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CHAPTER  1 


THE  RESEARCH  PROBLEM 


Introduction 

To  niset  national  defense  objectives  our  national 
security  policy  requires  that  the  United  States  remain 
militarily  strong.  A  strong  defense  is  partially  based  on 
the  development  of  high  technology  weapons.  In  contracting 
for  major  weapon  systems,  the  Department  of  Defense 
identifies  profit  as  the  basic  motivator  to  encourage 
contractors;  yet,  other  motivators  exist  and  need  to  be 
considered  if  we  are  to  arrest  the  decline  of  the  defense 
industrial  base  and  produce  quality  weapon  systems. 


Background 

The  companies  which  produce  the  weapon  systems, 
services,  and  supplies  necessary  to  meet  the  national 
objectives  of  the  United  States,  make  up  the  defense 
industrial  base.  From  1958  through  1975  the  number  of 


active  aerospace  subcontractors  decreased  from  over  6000  to 


under  4000.  In  this  same  period,  the  foundry  industry 
alone  experienced  240  net  closings.  Between  1970  and 
1975,  the  Air  Force  determined  that  the  number  of  sub¬ 


contractors  leaving  the  defense  area  more  than  doubled 
each  year  [5:129].  This  problem  has  become  so  large  that 
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the  House  Committee  on  Armed  Services  initiated  hearings  on 
September  17,  1930.  In  its  report  the  Committee  stated 
that  the  general  condition  of  the  defense  industrial  base 
has  deteriorated  and  is  in  danger  of  further  deterioration 
in  the  coming  years.  In  addition,  the  report  depicts  an 
industrial  base  crippled  by  declining  productivity  growth, 
aging  facilities  and  machinery,  shortages  in  critical 
material,  increasing  leadtimes,  skilled  labor  shortages, 
inflexible  government  contracting  procedures,  inadequate 
defense  budgets,  and  burdensome  paperwork  associated  with 
numerous  government  regulations  [2:1-5].  Indeed  Gansler 
(5:4)  concludes,  the  defense  industrial  base  is  becoming 
both  economically  inefficient  in  the  production  of  defense 
materials  and  strategically  unresponsive  to  a  production 
speedup  required  to  meet  an  emergency. 

To  meet  U.S.  National  defense  objectives,  the  Dod 
depends  on  private  industry  to  provide  tne  necessary  weapon 
systems,  services,  and  supplies.  To  this  end  the  House 
Committee  expressed  concern  for  present  policies  and 
procedures  which  DoD  uses  in  the  procurement  process 
(2:1).  According  to  the  Defense  Acquisition  Regulation 
(DAR),  profit  is  the  basic  motive  of  business;  low  average 
profit  rates  on  defense  contracts  are  detrimental  to  the 
public  interests;  and  an  effective  national  defense  in  a 
free  enterprise  economy  requires  that  the  best  industrial 
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capabilities  be  attracted  to  the  defense  market 
(22:3-308.1).  If  DoD  is  following  their  profit  policy  and 
yet  the  defense  industrial  base  continues  to  decline,  then 
we  can  infer  that  the  profit  policy  is  insufficient  to  keep 
the  industrial  base  strong. 

While  the  defense  industrial  base  is  declining,  there 
is  also  a  declining  need  for  the  quantity  of  goods. 
According  to  a  Rand  Corporation  study  (1:10-12),  since  the 
advent  of  the  atomic  age,  the  emphasis  on  weapon  systems 
acquisition  has  shifted  from  one  of  quantity  to  one  of 
quality.  The  requirement  for  massive  forces  designed  to 
deliver  large  quantitites  of  explosives  has  been  replaced 
with  highly  complex  and  sophisticated  weapon  systems.  The 
quality  emphasis  has  not  only  resulted  in  changes  in  design 
philosophy  but  has  also  created  many  problems  in 
procurement  management.  "It  is  a  systems  complexity  that 
entangles  the  development,  production,  and  maintenance 
problems  together,  creating  the  knottiest  problem  of 
procurement  management  (1:14J".  It  is  this  complexity  and 
entanglement  that  have  brought  about  the  establishment  of 
project  type  organizations  for  managing  the  procurement  of 
the  major  weapon  systems  (1:10-12). 

If  DoD's  profit  policy  is  unable  to  keep  the  defense 
industrial  base  from  declining  and  the  emphasis  is  on 
quality  instead  of  quantity,  then  contractor  motivation 
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industrial  oase 


becomes  important  in  sustaining  the  defense 
while  producing  quality  products.  Economic  theory  explains 
the  motivation  of  the  business  corporation  witn  tne  tneory 
of  the  firm  (5:  209-211  ).  Accordingly,  in  the  long  run  the 
entrepreneur  tries  to  maximize  profits,  where  profits  are 
defined  as  total  revenues  minus  total  costs.  Secondly,  the 


theory  assumes  a  free 

market. 

where 

the 

free  market  is 

defined 

as  a  market 

which 

exists 

witn  no  external 

controls . 

Under  these 

two  conditions 

tne 

market  operates 

under  perfect  competition  with  the  following  qualities: 

1.  Each  buyer  and  seller  is  unable  to  influence  price, 

2.  All  products  are  homogeneous, 

3.  Free  mobility  of  resources  such  that  one  can 
readily  move  in  or  out  of  the  market,  and 

4.  Perfect  knowledge  about  the  marketplace 
[6:212-215] . 

Tnese  conditions  are  generally  not  met  in  a  typical 
business  transaction  (3:44,  4:31).  when  considering  a 

business-government  transaction,  even  fewer  of  the 
conditions  are  met.  Finally,  for  major  weapon  systems 
acquisition  often  almost  all  conditions  are  violatej; 

1.  There  is  only  one  buyer  (a  monopsony), 

2.  There  is  generally  a  customized  product, 

3.  Entry  into  the  defense  market  is  very  difficult, 
and 


4 


■ 


wmmm 


4.  After  proposals  are  received,  only  the  ouyer  nas 

information  concerning  prevailing  prices  (3:44). 

As  a  result,  the  market  place  of  the  .major  weapon  system 
does  not  fit  the  competitive  situation  of  classical 
economics.  DoD  tries  to  compensate  for  tms  witn 

incentives  in  some  of  their  negotiated  contracts.  The  most 
important  function  of  the  proper  contract  design  is 
.motivation  of  the  contractor.  Depending  on  the 

circumstances  to  be  addressed  wnen  tne  contractual 
relationship  is  established,  there  are  different  types  of 
incentives  available,  including  fixed-price  incentives, 
cost-based  incentives,  multiple  incentives,  and  award 
incentives  (19:240-251). 

In  the  case  of  the  award  incentive,  the  award  fee 
provisions  in  federal  contracting  permit  government 
-monitors  to  unilaterally  determine  all  or  part  of  the 
contractor's  fee  on  the  basis  of  subjective  af ter-the-fact 
evaluations  of  contractor  performance.  While  F*  1973  award 
fees  represented  only  2.9  percent  of  the  net  value  of  DoD 
procurement  actions  and  about  0.5  percent  of  all  military 
prime  contracts,  Hunt  believes  the  award  fee  provides  a 
policy  choice  which  has  contracting  significance  far  in 
excess  of  its  modest  number  of  procurement  actions  (7:9). 

In  his  study  of  contractor's  response  to  the  award 
fee.  Hunt  (3:173-190)  found  that  contractor  personnel  nad: 
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1.  A  high  responsive  attitude, 


2.  3ood  communication  both  internally  to  their 
organization  and  externally  to  tne  buying  office, 

3.  An  uncomfortable  feeling  with  the  subjectivity  in 
spite  of  being  reasonably  satisfied  with  tne  outcome, 

4.  Motivation  from  the  award  fee,  and 

5.  Increased  leverage  for  the  Program  Manager  with 
his  own  management. 

In  addition.  Hunt  cautioned  that  the  value  of  the  award  fee 
depends  on  its  being  one  of  no  more  than  a  few  in  the 
contractor's  on-going  mix  of  contracts.  If  all  contracts 
were  an  award  fee,  we  can  infer  that  it  loses  the  special 
attention  of  the  management.  As  long  as  the  Award  Fee 
attracts  management's  attention. 


The  Award  Fee  method  is  adaptable  to  imaginative  uses 
in  acquisition  and  program  management,  such  as 
motivating  contractors  to  improve  their  quality 
assurance  programs  and  encouraging  cotractor 
performance  beyond  contract  minima,  when  sucn 
improvement  is  in  the  government's  interests  [3:53j. 

The  motivation  of  government  contractors  above  contract 

minima  is  a  key  concept  to  improving  the  quality  level. 


A  factor  in  the  use  of  the  award  fee  contract  is  the 
periodic  report  card  in  support  of  award  decisions, 
which  stimulates  intense  interest  in  program  progress 
by  top  management  of  both  parties.  It  continues  as  a 
stimulative  force  over  the  life  of  the  program  long 
after  the  glow  of  newness  is  gone  [13:257-2581. 
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According  to  a  study  completed  by  International 


Technology  Corporation  (Intec),  a  direct  relationship 
between  quality  and  profit  was  difficult  to  establish. 

Their  evidence  was  unable  to  support  the  contention  tnat 
profit  directly  affects  quality.  Finally,  their  findings 

indicated  that  motivational  rewards,  such  as  management 

recognition  of  good  workmanship  and  performance,  are  more 

effective  than  financial  rewards  (9:105-107).  As  a  result 

Intec  proposed  the  use  of  the  award  fee  for  quality. 

Hunt  and  Intec  identify  the  Award  Fee  as  a  possible 

method  of  incentivizing  quality,  which  is  one  of  the  chief 

concerns  in  the  production  of  major  weapon  systems.  In 
terms  of  mission  effectiveness  the  quality  of  material 

delivered  to  the  USAF  has  an  impact  on  military  readiness. 
General  Robert  T.  Marsh,  Commander  of  Air  Force  Systems 


Command  (AFSC), 

has  called 

for 

a  Quality 

Improvement 

Program  as  one 

way  to  meet 

the 

President' s 

mandate  to 

modernize  our 

defense  force 

in 

the  face 

of  limited 

resources.  To  seek  more  efficient  ways  of  acquiring  weapon 
systems,  General  Marsh  identified  Quality  Assurance  as  an 
area  in  which  we  can  achieve  significant  gains  with  minimum 
financial  investment  (15:1).  According  to  Leonard  and 
Sasser  (14:166),  quality  improvement  is  the  most  fruitful 
path  to  higher  productivity  and  competitive  success. 
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Higher  quality  and  lower  cost  can  be  acnieved 
prudent  investment  in  people,  product  design,  and 
improvement . 


tnrougn 


process 

In  summary,  DoD  relies  on  private  industry  to  develop 
the  magor  weapon  systems  required  for  defense  of  the 
country.  Yet,  the  defense  industrial  base  has  eroded  in 
spite  of  DoD's  policy  to  provide  adequate  profit.  Since 
the  marketplace  for  major  weapon  systems  is  not  a  free 
market,  DoD  cannot  rely  on  competition,  but  must  encourage 
the  defense  industry  with  incentives  through  negotiated 
contracts.  While  there  are  several  types  of  incentives, 
the  award  fee  incentive  can  be  used  to  encourage  contractor 
performance  beyond  the  minimum  specified,  i.e.,  beyond  the 
minimum  quality  specified.  Finally,  quality  of  material 
has  been  identified  as  an  area  which  could  improve  tne 
efficiencey  of  acquiring  weapon  systems.  The  Award  Fee  is 
one  method  to  achieve  this  efficiency. 

Problem  Statement 

To  meet  the  current  goal  of  AF5C  to  improve  quality  of 
material,  we  need  to  motivate  and  encourage  contractors  to 
deliver  a  better  quality  product.  In  a  noncompetitive 
market  the  award  fee  incentive  motivates  contractors  by 
involving  top  management  to  provide  leadership  for  quality 
improvement.  However,  there  are  no  clear  quality 
measurement  criteria  to  evaluate  the  contractor's 
performance  in  quality. 
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Review  of  the  Literature 


To  •  measure  quality,  there  must  be  a  consistent 
definition.  For  the  purpose  of  this  study  quality  will  be 
defined  by  the  DoD  definition,  "the  composite  of  material 
attributes  including  performance,  features,  and 
character istics  of  a  product  or  service  to  satisfy  a  given 
need  [  23  :  end  2  ]  .  " 

There  is  a  widely  held  view  among  the  public  tnat  the 
quality  of  United  States  manufactured  goods  has  declined. 
Lieutenant  General  Skantze  believes  the  decline  in  product 
quality  is  the  result  of  short-range  planning:  this  coupled 
with  the  improvement  of  the  next  period's  profit,  while 
concentrating  on  cost,  schedule,  and  performance  with  only 
minor  attention  to  quality  (20:12).  Feigenbaum  describes 
the  inefficiencies  associated  with  short-range  planning, 

Traditional  industrial  practices  have  created  the 
hidden  plant,  both  factory  and  office  sometimes 

amounting  to  as  much  as  40  percent  of  total  productive 

capacity,  which  is  used  to  replace  products  recalled 

from  the  field,  to  retest  and  reinspect  rejected 
units,  to  rework  unsatisfactory  parts,  or  to  maintain 
unduly  high  stocks  of  spare  parts  [4:23-24]. 

Yet,  to  Leonard  and  Sasser  quality  has  never  been  higher; 

for  example,  the  companies  in  the  field  of  computers, 

jeans,  and  telephone  service  produce  products  demanded 

throughout  the  world  because  of  their  excellence.  The 

perception  of  a  quality  decline  comes  from  foreign 

competitors,  which  have  taken  market  share  away  from 


9 


domestic  producers  because  of  the  superiority  of  the 
foreign  made  goods.  These  competitors  have  relentlessly 
pursued  quality  improvement  for  the  past  20  years  as  a  part 
of  national  strategy  to  build  an  export  economy.  In  both 
the  American  and  world  markets,  demand  for  quality  products 
has  increased.  As  demand  for  more  reliable,  durable,  and 
energy  efficient  products  increases,  so  does  the  demand  for 
high  quality  components,  parts  and  material  (14:163-164). 

Whether  the  quality  of  goods  in  the  U.S.  is  declining 
or  just  not  growing  as  fast  as  the  foreign  competitors,  the 
public  perceives  that  foreign  competitor's  goods  are 
superior.  In  general,  Japan  is  recognized  as  the  leader  in 
quality  manufactured  goods  (3:17,  10:163,  15:11).  The 
Japanese  quality  program  has  excelled  in  a  minimum  of  three 
areas:  massive  training  programs,  annual  quality 
improvement  programs,  and  upper  management  leadership  in 
quality.  Consequently,  Juran  feels  the  West  is  in  serious 
trouble  with  respect  to  quality  (9:9-11). 

According  to  Intec,  product  quality  is  specified  when 
a  technical  description  of  the  product  is  completed.  For 
major  weapon  systems  the  specifications  of  quality  begins 
in  the  conceptual  phase  by  determination  of  what  the 
performance  parameters  must  be.  Initially,  performance  is 
negotiated  and  traded  off  between  minimum  operational 
requirements  and  life  cycle  systems  cost.  During  full 
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scale  development  and  production,  the  quality  requirements 
are  firmly  established  as  technical  requirements. 
Determination  of  system  performance  requirements  are  made 
by  the  user  and  developing  command.  Deviations  from  these 
requirements  must  be  approved  (14:14).  This  infers  quality 
is  initially  determined  by  the  customer,  and  the  product 
must  meet  his  satisfaction. 

Quality  is  divided  into  "quality  of  design”  and 
"quality  of  conformance."  Juran  (13:8)  explains  quality  of 
design  as  a  difference  in  specification  for  the  same 
functional  use,  often  referred  to  as  grade  of  an  item. 
The  functional  need  can  be  illustrated  by  the  use  of  a 

Chevrolet  or  a  Cadillac  to  meet  the  same  purpose.  On  the 
other  hand,  quality  of  conformance  refers  to  the  accuracy 

of  the  product  meeting  the  design  specifications.  Juran 
also  states  that  quality  of  design  is  heavily  external  to 
the  company,  while  quality  of  conformance  is  internal. 

Both  require  consideration  to  meet  quality  in  tne  final 
product . 

Intec  (9:57)  further  divides  quality  into  a  third 
category,  "quality  of  the  contractor's  quality  management 
system",  which  is  the  quality  management  program.  With  the 
continuing  reduction  of  quality  personnel  there  has  been  3n 
increasing  reliance  on  the  concept  that  a  good  quality 

program  will  produce  conforming  material.  This  increasing 
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emphasis  has  led  to  the  concentration  of  emphasis  on  the 
contractor's  management  system. 

In  their  study,  Intec  found  no  significant  fault  with 
the  way  the  government  focuses  on  the  contractor's 
management  system  or  the  way  it  was  implemented.  Quality 
of  conformance  is  a  key  element  in  the  evaluation  of  the 
contractor's  management  system.  However,  quality  of  design 
was  not  perceived  to  be  part  of  the  quality  function. 
Policies  relative  to  design  quality  are  in  broad  sweeping 
concepts  that  relate  to  overall  system  performance 
including  life  cycle  cost  (9:57-58). 

The  measurement  of  quality  begins  by  determining  the 
contractor's  quality  of  design,  his  capability  to  produce 
as  specified,  and  the  effectivenss  of  his  quality  assurance 
program  to  assure  conformation  [13:31].  Since  quality  is 
determined  by  customer  satisfaction,  there  is  no 
universally  agreed  measure  of  quality.  Quality  is  usually 
specified  by  material  characteristic  dimensions  and  instant 
performance  parameters  (9:55).  These  dimensions  and 
parameters  become  the  objective  measure  for  Quality  of 
Design  and  Quality  of  Conformance. 

In  addition  to  specific  characteristics  of  a  product, 
quality  is  reflected  in  its  philosophy  for  business.  If 
the  philosophy  is  interpreted  in  the  quality  management 
system,  and  each  company  has  its  own  philosophy,  then  we 
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can  infer  that  each  company  has  a  unique  quality  management 
system.  Juran  (12:53)  points  out  that  the  quality  mission 
of  a  company  should  be  fitness  for  use,  which  is  an 
expansion  of  conformance  to  specifications,  standards, 
etc.  .'Ihen  companies  relegate  quality  to  an  individual  or 
department,  they  are  only  performing  part  of  the  task. 

To  achieve  quality  excellence  in  a  company,  A.  V. 
Feigenbaum,  President  and  Chief  Executive  Officer  of 
General  Systems  Company,  believes  companies  must  recognize 
two  fundamental  principles: 

1.  Quality  is  a  way  of  managing  the  organization, 

2.  A  new  form  of  competion  has  developed  involving 
company  effectiveness  in  quality  and  productivity 
management . 

Previously,  traditional  policies  were  oriented  to  customer 
satisfaction,  such  that  services  and  technical  assistance 
were  readily  available  to  customers.  Instead,  Feigenbaum 
promotes  quality  leadership  where  products  consistently 
perform  correctly  when  first  purchased,  and  that,  witn 
reasonable  maintenance,  will  continue  to  perform  with  nign 
reliability  and  safety  over  the  product  life.  Repairing 
the  product  when  it  fails  is  insufficient  to  meet  customer 
satisfaction  (  4  :  22-23  ).  Working  first  time  every  time 
becomes  the  essence  of  quality  improvement. 
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in  the  program  for  productivity  and  quality  improvement 
instituted  at  Public  Systems  Company,  /Jest ing house  Electric 
Corporation,  President  T.  J.  Murrin  believes  management's 
attitude  is  the  most  important  ingredient  in  making 
productivity  and  quality  improvement  programs  work.  Tnere 
are  many  factors  which  impact  operating  margins,  sucn  as 
inflation,  interest  rates,  government  regulations,  and 
recession  sensitive  markets.  These  are  external  factors 
over  which  management  has  little  control.  Management  needs 
to  pay  greater  attention  to  the  aspects  which  they  are 
able  to  control,  i.e.,  technology,  money,  time,  assets,  and 
■quality.  Better  management  of  resources  means  improved 
performance  no  matter  which  way  the  economy  goes  (17:14). 

Generally,  quality  improvement  must  start  with  top 
management  (11:11;  17:14;  4:22).  The  design  and  production 
of  high  quality  goods  is  not  just  the  quality  manager's 
problem,  but  also  the  general  manager's  problem.  Juran 
identifies  quality  activities  of  two  types:  managerial 
activity  and  technological  activity.  At  the  managerial 
level  policy  is  formed,  objectives  set,  people  selected  and 
trained,  people  motivated,  and  control  measures  set.  On 
the  other  hand,  technologocial  activities  include  the 
quality  of  such  things  as  product  design,  specifications, 
manufacturing  planning,  instrumentation,  production, 
inspection  and  test,  selling,  and  service  (13:35).  Leonard 
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and  Sasser  (14:168)  found  that  companies  with  a  successful 
quality  program  included: 


1.  Top  management's  strategic  support, 

2.  Organizational  analysis, 

3.  Responsibility  resting  with  the  total  organization, 
and 

4.  Open  participation  by  all  employees. 

From  this  we  can  infer  that  a  quality  improvement  program 
should  be  aimed  at  the  managerial  activity  level.  w’itnout 
changes  in  policies  and  objectives,  people  within  the 
organization  will  not  be  fully  motivated  to  acnieve  better 
qual ity . 

The  Quality  Horizons  Report  of  1979  raised  a  question 

concerning  the  improvement  of  quality  for  major  systems 
acquisition.  The  study  group  foresaw  a  need  to  identify 
changes  with  potential  to  improve  end  items  quality  in 

field  use,  make  contractors  more  responsible  for  their 
products,  make  more  effective  use  of  resources,  and  apply 
appropriate  commercial  practices.  Two  of  their  findings 

were : 

1.  Attainment  of  field  quality  is  a  function  of  the 

interest  and  priority  placed  on  quality  by  top 
managers,  and 
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2.  Product  Assurance  cannot  be  inspected  into  any 
product.  Nevertheless,  AFSC  places  more  emphasis  on 
conformance  verification  than  attempting  to  influence 
product  quality  through  design,  process  control,  and 
test  planning  early  in  the  program  life  cycle. 

High  levels  of  quality  are  obtained  in  the  commercial 
sector  where  top  management  demands  product  quality  or 
where  competitive  market  pressures  force  product  quality. 
Since  the  market  for  major  weapon  systems  does  not 
correspond  to  the  free  competitive  market,  top  management 
must  be  motivated  to  become  involved  (18:1-2). 

Intended  to  establish  uniform  procurement  policies,  the 
DAR  devotes  Section  XIV  to  quality  assurance  organizational 
roles  and  responsibilities.  The  DoD’s  primary  principle  of 
quality  policy  in  the  administration  of  contracts  is  the 
contractor's  responsibility  concept  coupled  with  the  DoD 
requirement  for  some  type  of  management  system  (9:70). 

The  most  detailed  management  system  is  outlined  by 
MIL-Q-9858A,  Quality  Program  Requirements,  which  is 
required  for  purchases  of  complex,  critical  items  defined 
by  military-federal  specifications.  Complex  items  have 
quality  characteristics  not  wholly  visible  in  the  end  item 
for  which  contractual  conformance  must  be  progressively 
established.  In  the  event  of  failure,  a  critical  item 
will  result  in  injured  personnel  or  jeopardize  a  military 
mission  (21:14-101). 


In  summary,  quality  can  be  considered  as  quality  of 


design,  quality  of  conformance,  and  quality  of  the 
contractor's  management  system.  From  all  information, 
management  provides  the  leadership  to  influence  quality.  A 
progressive  and  improving  attitude  towards  quality  must 
come  from  the  top.  While  quality  begins  with  quality  of 
design,  DoD  emphasizes  quality  of  conformance;  and 
MIL-Q-9358A  is  the  most  detailed  quality  management  program 
used  to  assure  compliance  with  the  contract  requirements. 

Research  Objective 

The  research  objective  is  to  analyze  the  quality 
evaluation  criteria  which  have  been  used  on  Air  Force  Award 
Fee  programs  to  incentivize  quality  and  to  recommend  a  set 
of  quality  evaluation  areas  for  future  use  with  all  Award 
Fee  contracts  for  Air  Force  weapon  systems. 

Research  Question 

1.  Are  the  evaluation  criteria  in  Air  Force  Weapon 
Systems  Procurement  used  to  incentivize  quality  on  Award 
Fee  contracts  consistent  with  DAR  requirements? 

2.  How  do  contractors  involved  with  the  Award  Fee 
perceive  the  award  fee  concept  and  the  associated  quality 
evaluation  criteria? 

3.  How  do  government  quality  managers  involved  with  the 
Award  Fee  perceive  the  award  fee  concept  and  the  associated 
quality  evaluation  criteria? 
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CHAPTER  2 


RESEARCH  METHODOLOGY 

Award  Fes  contracting  is  a  management  tool  which  uses 
subjective  evaluation  of  performance  as  a  basis  for 
determining  contractor  compensation.  The  Air  Force  uses 
Award  Fees  to  help  achieve  goals  in  major  system  and 
subsystem  acquisitions.  The  Quality  Horizon  Study  (18:90) 
recommended  the  Award  Fee  as  a  method  to  motivate  enhanced 
product  quality.  As  a  result,  this  research  was  undertaken 
to  evaluate  the  application  of  Award  Fees  in  relation  to 
fee  theory  and  other  considerations  with  the  plan  to  offer 
recommendations  for  future  research  and  practice. 

The  previous  chapter  described  the  current  approach  to 
a  quality  product  in  the  weapon  acquisition  process.  To 
motivate  contractors  to  provide  enhanced  product  quality, 
quality  measurement  criteria  are  needed.  However,  before 
criteria  can  be  selected,  the  areas  of  evaluation  must  be 
determined.  Justif ication  of  this  research  existed  based 
on  the  following  perspectives:  first,  observations  of  the 

quality  system  as  viewed  by  the  author;  second,  the  Air 
Force  Business  Research  Management  Center  identified  the 
need  for  this  research;  finally,  Aeronautical  Systems 
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Division  (ASD)  had  expressed  concern  about  quality 
evaluation  criteria  for  the  award  fee  contracts. 

Even  though  ASD  is  the  only  division  within  AFSC  which 
is  currently  using  the  quality  incentives  with  the  Award 
Fee  contract,  the  research  will  apply  to  all  DoD  agencies 
governed  by  the  DAR.  The  uniform  procurement  requirement 
contained  in  the  DAR  includes  quality  requirements  which 
apply  to  all  contractors  working  on  Department  of  Defense 
contracts . 


Scope  of  the  Research 

The  Award  Fee  contract  is  a  subjective  tool,  yet 


evaluation  areas  must  be  defined  and  communicated  to  the 


contractor.  This  research  did  not  attempt  to  evaluate  the 
policies  and  procedures  that  sat  quality  requirements, 
however,  this  research  was  directed  at  standardizing 
quality  evaluation  areas  for  use  with  the  Award  Fee 
incentive.  The  specific  objectives  of  the  project  are  to 

(1)  compare  the  quality  evaluation  criteria  used  in  the  two 
Award  Fee  plans  with  quality  areas  listed  in  MIL-3-9853A, 

(2)  obtain  the  response  of  contractor  personnel  to  the 
Award  Fee  and  its  associated  quality  evaluation  criteria, 
and  (3)  obtain  the  response  of  government  managers  to  the 
award  fee  and  its  associated  quality  evaluation  criteria. 
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Population  of  Interest 

While  the  universe  of  Award  Fee  contracts  is  large 

within  the  Air  Force  Systems  Command,  the  population  of 
interest  narrows  to  the  set  of  Award  Fee  contracts  whicn 
were  used  to  incentivize  quality.  Within  ASD  there  were 
two  instances  where  the  Award  Fee  was  used  to  incentivize 

quality.  Therefore,  the  census  for  this  study  is  tne  Award 
Fee  under  contract  F33657-31-C-2100  on  the  Advanced  Concept 
Ejection  Seat  (ACES  II)  and  the  Award  Fee  plan  under  tne 

Request  for  Proposal  ( RFP )  F33657-32-R-0087  for  the  AW/APR 

38  Wild  Weasel  Performance  Update  Program  (PUP). 

In  the  case  of  the  AN/APR  38  the  Award  Fee  is  to  be 

included  in  a  future  production  contract.  While  we  will 
not  have  all  the  results  of  a  completed  program,  we  nave 
partial  results  to  analyze. 

Research  Question  1 

Are  the  evaluation  criteria  in  Air  Force  Weapon  Systems 
Procurement  used  to  incentivize  quality  on  Award  Fee 
contracts  consistent  with  DAR  requirements? 

Data  Collection  Plan 

Data  for  Research  Question  1  was  collected  oy 
observation  from  contract  files  at  ASD.  A  census  of  tne 
population  was  taken  from  the  Award  Fee  plans  w.oicn  ASD 
used  in  the  two  instances  to  incentivize  quality, 
F33657-31-C-2100  for  ACES  II  and  RFP  F33657-32-R-0067  for 
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Wild  Weasel  P'JP.  As  data  was  collected  from  tne  Awara  tee 
plans  general  requirements  were  placed  into  one  of  five 
categories  within  MIL-3-9853A.  For  specific  requir  e.nents 
which  are  peculiar  to  tne  product,  the  data  was  noted, 
collected  as  other  pertinent  information,  and  not  used 
further  since  the  objective  is  to  obtain  standardized 
evaluation  areas. 

Data  Analysis  Plan 

Data  was  compared  and  analyzed  to  quality  areas  in 
MIL-Q-9853A  (see  Appendix  A).  Request  by  DAR  for  items 
which  are  complex,  critical,  and  defined  by 
military-federal  specifications,  MIL-Q-9358A  embraces  the 
broadest  and  most  detailed  evaluation  for  the  contractor's 
quality  system.  For  this  study,  MIL-Q-9358A  (21:2-3)  was 

divided  into  five  categories: 

1.  Quality  Program  Management. 

a.  Organization. 

b.  Initial  Quality  Planning. 

c.  Work  Instructions. 

d.  Records. 

e.  Corrective  action. 

2.  Facilities  and  Standards. 

a.  Drawings,  Documentation  and  Cnanges. 

o.  Measuring  and  Testing  Equipment. 
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3.  Control  of  Purchases. 

a.  Responsibility. 

b.  Purchasing  Data. 

4.  Manufacturing  Control. 

a.  Material  and  Materials  Control. 

b.  Production  Processing  and  Fabrication. 

c.  Completed  Item  and  Inspection  Testing. 

d.  Handling,  Storage  and  Delivery. 

e.  Nonconforming  Material. 

f.  Indication  of  Inspection  Status. 

5.  Coordinated  Government/Contractor  Actions. 

a.  Government  Inspection  at  Subcontractor  or 
Vendor  Facilities. 

b.  Government  Property. 

Data  included  in  the  five  categories  was  so  noted,  and  data 
outside  of  the  categories  was  noted  and  collected  as  other 
pertinent  data. 

Research  Question  2 

How  do  contractors  involved  with  tne  award  fee  perceive 
the  award  fee  concept  and  the  associated  quality  evaluation 
criteria? 

Data  Collection  Plan 

Data  for  Research  Suestion  2  was  collected  oy  interview 
from  the  contractor's  program  manager  and  staff  involved 
with  the  award  fee  under  F33657-31-C-2100  and  RFP  F3J657- 
82-R-0067.  A  census  of  the  population  included  McDonald 
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Aircraft  Company,  Wild  tfeasel  PUP;  Douglas  Aircraft 
Company,  ACES  II;  and  Weber  Aircraft,  ACES  II.  While  Weber 
Aircraft  did  not  have  an  Award  Fee  contract,  tney  were  tne 
follower  on  ACES  II  in  the  leader- follower  program.  As  tne 
follower  Weber  had  an  input  into  the  award  fee  evaluation 
process.  Interviews  will  be  conducted  as  discussions  of 
the  "Questions  for  Contractors"  which  are  contained  in 
Appendix  3.  In  that  interview  guide  the  first  eignt 
questions  were  used  by  Hunt  in  his  study  (3:191-192).  The 
interview  encouraged  interviewees  to  share  a  broad  range  of 
Award  Fee  information  while  addressing  the  specific 
topics.  To  promote  a  frank  discussion  interviews  were 
conducted  without  attribution. 

Data  Analysis  Plan 

Data  was  compared  and  analyzed  for  common  responses  and 
exceptions  to  the  Award  Fee  and  standard  evaluation  areas 
for  quality. 

Research  Question  3 

How  do  government  quality  managers  involved  with  the  Award 
Fee  perceive  the  Award  Fee  concept  and  the  associaced 
quality  evaluation  criteria? 

Data  Collection  Plan 

Data  for  Research  Question  3  was  collected  by  interview 
of  the  census  of  the  population  of  the  government  managers 
involved  with  the  Award  Fee  under  F33657-31-C-2100  and  RF? 
F33657-32-R-0067 .  Interviews  were  conducted  as  discussions 
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of  the  "Questions  for  Government  Managers,"  wmcn  ere 
contained  in  Appendix  B.  In  that  interview  guide  the  first 
eight  questions  were  used  by  Hunt  in  his  study 
(3:191-192).  The  interview  encouraged  interviewees  to 
share  a  broad  range  of  Award  Fee  information  wnile 
addressing  the  specific  topics.  To  promote  a  franx 
discussion  interviews  were  conducted  without  attribution. 
C3ta  Analysis  Plan 

Data  was  compared  and  analyzed  for  common  responses  and 
exceptions  to  the  Award  Fee  and  standard  evaluation  areas 
for  quality. 

Criteria  Test 

With  tne  three  questions  there  are  six  possible 
combinations  of  outcomes.  The  strongest  indicator  of 
standardized  quality  evaluation  areas  will  be  tne  data  from 
question  3,  since  they  are  experts  involved  with  qjality 
Award  Fees  and  are  interested  in  the  government  position. 
Second  is  question  2,  because  the  data  will  favor  the 
contractor's  viewpoint  of  the  best  way  to  gudge 
themselves.  Finally,  the  data  for  researcn  question  1  is 
the  weakest  indicator,  since  it  will  be  data  developed  ac 
one  point  in  time  and  does  not  have  the  ability  to  evaluate 
what  was  good  or  bad  about  tne  evaluation  areas. 

The  best  situation  would  show  all  three  questions  agree 
to  the  quality  areas  to  be  included.  However,  it  data  on 
two  sets  of  the  three  questions  are  agreeable,  then  tne 
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research  objective  of  determining  standard  quality 
evaluation  areas  for  the  quality  Award  Fee  incentive  will 
be  considered  possible. 
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CHAPTER  3 

FINDINGS 

The  specific  objectives  of  the  project  were  to  (1) 
compare  the  quality  evaluation  criteria  used  in  the  two 
Award  Fee  plans  with  quality  areas  listed  in  MIL-Q-9358A, 
(2)  obtain  the  response  of  contractor  personnel  to  the 
Award  Fee  and  the  quality  evaluation  criteria,  and  (3) 
obtain  the  response  of  government  managers  to  the  Award  Fee 
and  the  quality  evaluation  criteria. 

Comparison  of  Award  Fee  Plans 

There  are  two  instances  at  ASD  where  quality  evaluation 
areas  were  heavily  weighted  in  the  Award  Fee  Plan.  Under 
ACES  II,  contract  F33657-31-C-2100 ,  the  Quality  Assurance 
evaluation  criteria  constituted  30%  of  the  total  weight 
during  the  first  evaluation  period  and  25%  of  the  total 
weight  during  the  second  evaluation  period.  Under  Wild 
Weasel  PUP,  RFP  F33657-82-R-0067 ,  Qual i ty/Pr oduc ibi 1 i ty  and 
Product  Assurance  constituted  75%  of  the  total  weight  of 
the  Award  Fee. 

Each  program  had  different  goals,  so  their  criteria  was 
designed  to  meet  those  goals.  Also  areas  in  MIL-Q-9858A 
are  not  mutually  exclusive,  hence  they  are  redundant  and 
overlap.  All  evaluation  criteria  which  were  used  are 
contained  in  Appendix  C. 
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Except  for  a  few  arear  which  fit  into  specific 
categories,  all  evaluation  criteria  fell  into  the  Quality 
Progra.n  Management  category.  In  the  case  of  ACES  II, 
Facilities  and  Standards  included  the  evaluation  criteria 
of  the  calibration  system  and  manufacturing/tooling 


inspection . 

For  the 

category  of  Control  of 

Purchases 

the 

evaluation 

cr iter ia 

included  supplier 

rating 

and 

selection . 

Under  th‘ 

e  Manufacturing  Control 

ca  tegor  y 

the 

evaluation 

criteria 

included  major/critical 

product 

or 

system  deficiencies  and  source  inspection.  All  remaining 
criteria  fell  under  the  Quality  Program  Management  category 
and  included  procedures  evaluation,  quality  planning, 
corrective  action,  trend  analysis,  timeliness/ adequacy  of 
corrective  action,  work  instructions,  schedule  completion 
and  cost  of  quality. 

The  planned  evaluation  areas  for  'Wild  Weasel  were 
developed  after  ACES  II.  The  criteria  was  detailed  and  not 
as  easily  classified  under  MIL-Q-9358A  categories.  Again 
Control  of  Purchases  was  included  by  evaluating  the  control 
of  suppliers.  Manufacturing  Control  was  evaluated  by 
in-process  preventive  quality  control  and  delivered  product 
quality.  All  of  the  other  characteristics  fell  under  the 
Quality  Program  Management  category  which  included  quality 
of  design,  product  assurance,  and  adherence  to  management 
system  requirements. 
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In  summary,  the  majority  of  the  criteria  used  were 
subjective  evaluations  of  management  of  quality. 
Additional  criteria  evaluated  control  of  purcnases  and 
manufacturing  control. 

Results  of  Contractor's  Interviews 
Reported  here  are  the  results  of  in-depth  interviews 
with  three  contractors  associated  with  the  Award  Fee  on 
ACES  II  and  Wild  Weasel  PUP.  A  total  of  14  individuals, 
mainly  program  managers  and  staff,  were  interviewed. 
Interviews  ranged  in  length  from  one  hour  to  approximately 
three  hours.  On  two  occasions  more  than  one  representative 
of  a  contractor's  organization  participated  together  in  the 
discussion,  which  regularly  included  reviews  of 
contractor-supplied  documents  relevant  to  the  subjects  of 
discussion.  All  responses  shown  are  summaries  of  interview 
questions  and  quotations  are  without  attribution. 

1.  What  effect  did  Award  Fee  have  on  Organizational  and 
Staffing  Patterns? 

From  the  contractor's  perspective  there  was  little 
extra  work  required.  ACES  II  used  no  additional  staff 
and  Wild  Weasel  PUP  anticipates  no  additional  staff.  A 
matrix  organization  should  be  adequate  to  handle  all 
program  requirements  in  the  limited  time  the  award  fee 
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is  in  effect.  While  there  were  no  additional  personnel 
required,  the  additional  workload  did  require  more  time 
and  effort.  In  the  case  of  ACES  II  there  was 
considerable  time  and  effort  by  the  quality 
department.  Extensive  and  detailed  planning  were 
required  to  meet  the  government  quality  system 
requirements.  The  follower  company  was  not  involved 
with  government  contracting  for  ten  years  and  found  the 
leader's  quality  department  to  be  very  responsive. 

A  major  concern  for  all  companies  involved  was  to 
understand  what  is  necessary  and  to  perform  according 
to  contract.  From  a  program  manager's  point  of  view  to 
exceed  contract  requirements  increases  the  cost  to  the 
government,  contractor  or  both. 

2.  What  Information  is  given  contractors  on  Government 
Award  Fee  Plans? 

Award  Fee  plans  were  comprehensive  and  straignt- 
forward.  The  System  Program  Office  (SPO)  solicited 
comments  and  some  performance  factors  were  affected  in 
negotiations.  Each  plan  provided  evaluators  by  office 
symbol  and  organization,  number  of  evaluation  periods, 
and  time  phasing  of  the  evaluation  periods. 

3.  Do  the  Contractors  contribute  Input  to  Award  Fee 
Evaluations? 

As  part  of  the  Award  Fee  plans  the  contractors  were 
provided  the  opportunity  to  present  input  to  the  Fee 
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Determining  Official  (FDO).  While  the  Wild  Weasel  nas 
not  progressed  to  that  stage,  the  ACES  II  contractor 
did  take  advantage  of  his  opportunity  to  meet  with  the 
FDO  prior  to  the  3ward.  As  a  result  of  the  meeting  and 
his  explanation  of  the  company's  effort,  the  Award  Fee 
was  increased. 

4.  What  Feedback  is  Given  Contractors  on  their  Award  Fee 
Evaluations? 

Feedback  is  expected  if  progress  is  desired.  In 
the  case  of  ACES  II  the  contractor  did  receive  feedback 
for  both  evaluation  periods.  As  a  result  of  the 
feedback  from  the  first  period,  Douglas  Aircraft 
personnel  believed  the  same  approach  should  be  used  in 
the  second  period. 

b.  How  well  do  you  understand  the  Award  Fee  Objective? 

The  contractor's  personnel  understood  the  basic 
objectives  of  the  Award  Fee  in  each  situation.  For 
ACES  II  the  objective  was  to  motivate  the  leader  to 
provide  training  and  the  transfer  of  knowledge  to  the 
follower.  For  Wild  Weasel  PUP  the  objective  was  to 
motivate  the  contractor  to  meet  Design  to  Cost  goals. 
The  Award  Fee  is  the  government's  attempt  to  create  an 
incentive  "over  and  above"  the  contract  and  to  reward 
for  excellence  of  performance.  One  objective  is  to 
receive  a  high  quality  product  or  higher  than 


specified . 
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Even  though  contractor  personnel  understood  tne 
basic  objective  for  each  program,  they  felt  uneasy 
about  the  unilateral  determination  of  the  Award  Fee. 

6.  How  does  the  Award  Fee  affect  Organizational  Processes? 

Each  program  is  relatively  minor  to  the  overall 
defense  business  of  the  corporation.  As  a  result,  the 
Award  Fee  has  little  effect  on  organizational  processes. 
If  anything,  communication  improves.  However,  it  is 
the  job  of  each  person  involved  on  the  program  to  know 
the  line  and  functional  organization  in  order  to 
discuss  and  resolve  problems.  The  job  of  management  is 
to  focus  attention  and  to  resolve  identifiable 
problems;  and,  while  management  was  concerned,  tney 
displayed  no  unusual  attention  because  of  the  Award  Fee. 

In  the  case  of  ACES  II  the  product  had  high 
visibility,  since  it  was  a  life  support  system.  Mean¬ 
while,  the  follower  believed  the  Award  Fee  provided 
extra  incentive  to  communicate,  but  there  was  not  extra 
management  attention. 

7.  What  are  the  effects  of  Award  Fee  on  Program  Outcomes? 

Most  people  believe  the  Award  Fee  can  only  benefit 
the  contract,  since  it  is  another  motivator.  At  worse, 
it  has  no  effect.  For  the  leader-follower  situation, 
the  contractor's  personnel  viewed  the  fee  as  a  payoff 
for  the  transfer  of  knowledge  at  the  expense  of  the 
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leader.  The  amount  of  business  which  would  be  lost  in 
the  future  years  far  exceeds  the  amount  of  the  Award 
Fee.  Also,  the  Air  Force  must  maintain  enougn  business 
to  support  the  two  contractors. 

In  the  case  of  the  Wild  Weasel,  it  is  too  early  to 
determine  the  program  outcome,  but  the  contractor's 
personnel  believe  it  will  not  hinder  the  program.  As 
long  as  the  proper  criteria  are  used,  it  should  nelp 
the  program. 

8.  Is  the  Award  Fes  used  in  Subcontracting? 

While  one  company  did  have  some  Award  Fee  subcon¬ 
tracts,  they  are  rare  and  most  personnel  have  no 
knowledge  of  them. 

9.  What  are  some  Strengths  and  Weaknesses  of  the  Award 
Fee  Method  of  Acquisition? 

The  primary  strength  comes  from  the  additional 
profit  incentive  allowing  management  to  focus  attention 
on  desired  areas  of  interest.  As  an  incentive  tool,  it 
should  provide  more  than  would  otherwise  be  achieved. 

From  the  contractor's  viewpoint,  the  Award  Fee's 
only  weakness  comes  from  the  subjective  evaluation 
criteria.  In  the  case  of  ACES  II,  the  contractor's 
personnel  would  prefer  an  Award  Fee  based  on  product 
enhancement  instead  of  a  leader-follower  situation. 
For  Wild  Weasel  the  Award  Fee  is  not  definite,  it  is  a 
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carrot  that  could  disappear  from  the  later  production 
contract . 

10.  What  are  the  Strengths  and  Weaknesses  of  tne  Award  Fee 
Method  for  Stimulating  Better  Quality? 

The  Award  Fee  is  structured  differently  from  other 
incentives,  but  the  idea  of  motivating  the  contractor 
to  achieve  a  specific  goal  remains  the  same.  Contrac¬ 
tor  personnel  are  uneasy  about  the  subjective  evaluation 
criteria  preferring  objective  criteria  in  incentive 
arrangements.  While  the  Award  Fee  is  a  positive 
motivator,  money  does  not  buy  better  quality.  Quality 
must  be  bid  into  the  job  with  the  proposal  prior  to  the 
design.  A  quality  conscious  company  will  strive  to 
improve  quality,  and  the  Award  Fee  does  not  necessarily 
increase  the  quality  consciousness  of  the  company. 
However,  the  Award  Fee  does  have  an  advantage  over  the 
incentive  fee,  since  the  incentive  fee  aims  at  control¬ 
ling  cost  which  may  conflict  with  quality  and  cause 
tradeoffs.  The  best  place  to  use  an  Award  Fee  is  trying 
to  implement  a  quality  consciousness,  since  regulations 
merely  induce  compliance. 

11.  What  Quality  Criteria  should  be  used  in  the  Evaluation 
Process? 

Quality  is  achieved  by  knowing  the  customer's 
requirements  and  fitting  the  product  to  its  intended 
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use.  The  quality  conscious  company  knows  that  good 
quality  is  good  business.  To  this  end  one  must  look 
for  management’s  attitude  and  quality  control's 
attitude.  Some  factors  to  consider  are  participation 
in  design  review;  continued  configuration  management; 
integration  of  subassemblies  into  the  weapon  system; 
an  independent  quality  system  reporting  to  an  executive 
division;  the  quality  of  the  people,  their  knowledge, 
and  experience;  evaluation  of  vendor's  quality  and 
compliance  to  requirements;  cost  related  to  quality 
in  the  form  of  scrap,  repair,  and  rework;  management 
attention;  examination  of  negative  trends  and  correc¬ 
tive  action;  and  proper  manning  of  the  organization. 

12.  What  influence  did  the  Award  Fee  have  on  the  design  of 
product? 

For  ACES  II  the  design  was  completed  prior  to  the 
Award  Fee.  Consequently,  there  was  no  influence  on  the 
design. 

In  contrast,  the  Wild  Weasel  PUP  is  in  the  develop¬ 
ment  stage.  Even  in  this  case,  contractor  personnel 
claim  the  design  was  too  far  along  to  be  heavily 
influenced  by  the  Award  Fee.  The  primary  incentive  to 
produce  a  good  design  was  the  competition  to  win  the 
contract  over  the  contractor's  competition. 
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Since  specifications  usually  set  the  requirements, 
both  contractors  believe  the  Award  Fee  will  influence 
the  design  of  a  product,  if  criteria  are  detailed  and 
early  enough. 

13.  What  problems  occurred  with  the  Quality  Criteria 
associated  with  the  Evaluation  Process? 

With  the  exception  of  subjectiveness  of  criteria, 
the  contractor  personnel  had  no  complaints  about  tne 
quality  criteria.  To  them  the  quality  criteria  were  a 
contractual  requirement  which  had  to  be  accomplished. 

14.  What  feedback  does  the  contractor  receive  about  the 
product  in  the  field? 

Contractor  personnel  had  no  complaints  about  data 
from  the  field.  Service  reports  and  maintenance  data 
were  available  and  used. 

Results  of  Government  Manager's  Interviews 
Reported  here  are  the  results  of  in-depth  interviews 
with  government  managers  associated  with  the  Award  Fee  on 
ACES  II  and  Wild  Weasel  PUP.  A  total  of  12  individuals, 
mainly  program  managers  and  staff,  were  interviewed. 
Interviews  ranged  in  length  from  one  hour  to  approximately 
two  hours.  On  one  occasion,  two  government  managers 
participated  together  in  the  discussion.  All  responses 
shown  are  summaries  of  interview  questions  and  quotations 
are  without  attribution. 
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1.  What  effect  did  Award  Fee  have  on  Organizational  and 
Staffing  Patterns? 

From  the  government  perspective  there  was  little 
effect  on  the  organizational  structure  and  staffing 
patterns.  System  Program  Offices  are  set  up  under  a 
matrix  type  organization  with  support  provided  by  a 
home  office  within  ASD.  The  same  personnel  provided 
representation  whether  or  not  there  was  an  Award  Fee. 
Manpower  limitations,  generally,  prevented  additional 
people  from  being  assigned.  Considering  the  lifespan 
of  the  SPO,  the  Award  Fee  involved  a  relatively  short 
period  of  time  in  that  lifespan. 

Even  though  the  program  generally  operated  with 
existing  personnel,  there  was  an  increase  in  the  work¬ 
load.  The  Award  Fee  was  a  new  task  requiring  learning 
and  planning.  Teams  had  to  be  formed,  criteria  setup, 
and  problems  anticipated.  Evaluations  included  support 
from  resident  government  personnel  at  the  contractor's 
facility  as  well  as  the  SPO's  evaluation  team.  While 
some  data  was  subjective,  it  needed  to  be  analyzed.  As 
the  emphasis  shifted,  changes  were  made  to  the  evalua¬ 
tion  criteria.  If  members  of  the  evaluation  team  had 
negative  feelings  toward  the  Award  Fee,  they  had  to  be 
motivated;  and  the  contractor  had  to  be  unusually 
responsive  to  requests  from  the  government. 
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There  were  positive  benefits  in  tne  form  of  in¬ 
creased  management  attention  and  greater  communication 
by  program  participants.  During  the  early  phases 
management  required  briefings  on  the  Award  Fee  plans 
and  objectives.  As  a  result  closer  ties  were  formed 
within  the  matrix  organization  and  earlier  in  the 
program.  In  addition,  more  support  was  required  from 
the  resident  organization  providing  feedback  to  tne 
program  office.  Finally,  there  was  greater  interfacing 
with  the  contractor. 

2.  What  Information  is  given  contractors  on  Government 
Award  Fee  Plans? 

The  Award  Fee  plan  is  a  comprehensive  breakdown  of 
the  performance  factors  and  the  plan  for  evaluation. 
Contractors  can  have  some  input  during  negotiation, 
however,  the  basic  plan  is  provided  by  the  government. 
Performance  factors  are  divided  into  major  areas. 
However,  the  contractor  is  not  provided  a  detailed 
breakdown  of  the  scoring  procedure  and  the  criteria 
which  the  evaluator  uses.  As  a  result,  there  is 
flexibility  to  change  emphasis  as  needed,  since  the 
process  is  iterative.  From  the  general  areas  the 
contractor  is  able  to  focus  attention  on  the  objectives 
and  should  be  able  to  approximate  the  Award  Fee. 
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The  plan  identifies  the  number  of  evaluation  periods 
and  the  evaluators  by  the  office  symbol  and  the  organi¬ 
zation.  Names  are  not  associated  in  the  early  stages 
due  to  potential  personnel  changeover.  The  key  people 
are  usually  the  same,  although  new  personnel  may  be 
introduced  during  each  period.  Each  area  of  expertise 
should  be  represented  and  the  areas  should  be  germane 
to  the  performance  factors  rated.  While  evaluation 
periods  are  identified,  the  resident  government  person¬ 
nel  may  perform  continuous  observations  during  the 
per iod . 

3.  Do  the  Contractors  contribute  Input  to  Award  Fee 
Evalua  tions? 

In  these  two  programs  the  contractors  have  tne 
opportunity  to  brief  the  FDO.  In  the  case  of  the  Wild 
Weasel  the  program  has  not  progressed  that  far,  but  tne 
Award  Fee  plan  has  the  provision  for  the  contractor  to 
offer  self-evaluation.  It  is  an  opportunity  to  be  vocal 
and  explain  how  the  contractor  met  the  performance 
factors  and  exceeded  the  requirements.  As  a  result, 
the  contractor  must  brief  a  high  level  official 
and  address  issues  not  normally  brought  before  higner 
management  within  the  Air  Force. 
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4.  What  Feedback  is  Given  Contractors  on  their  Award  Fee 
Evaluations? 

Feedback  varies  for  each  program,  but  formal  and 
informal  feedback  was  provided  to  the  contractors.  To 
improve,  it  is  necessary  to  provide  feedback  on  lessons 
learned.  While  formal  briefings  were  held  for  each 
evaluation  perod,  it  is  questionable  w'netner  the 
evaluation  team's  report  should  be  provided  to  the 
contractor.  The  resident  government  representative 
must  maintain  a  good  working  relationship  with  the 
contractor  and  candid  opinion  could  be  withheld.  The 
majority  of  informal  quality  feedback  would  come  from 
the  resident  Quality  Assurance  Representative  ( QAR) . 
In  any  event  care  must  be  exercised  with  informal 
feedback  to  avoid  constructive  changes  which  would 
entitle  the  contractor  to  equitable  adjustment  under 
the  Changes  clause  of  the  contract. 

5.  How  well  do  you  understand  the  Award  Fee  Objective? 

Each  program  had  a  specific  objective,  and  response 
to  this  question  showed  a  good  understanding  of  the 
Award  Fee  objectives  and  process  for  each  program.  The 
objective  of  ACES  II  was  to  motivate  the  leader  to 
orient  the  follower  to  provide  a  quality  product  in  a 
reasonable  time  consistent  with  the  leader's  past 


39 


performance.  For  the  Wild  Weasel  PUP  the  objective 
was  to  motivate  the  sole  source  contractor  to  control 


production  and  life  cycle  costs  without  sacrificing 
quality  of  the  product.  Each  of  these  has  a  strong 
incentive  to  do  things  early  in  the  contract  to  cause 
quality  to  be  designed  into  the  system.  To  accomplish 
the  task  of  achieving  a  quality  product  requires 
criteria  which  can  be  measured. 

6.  How  does  the  Award  Fee  affect  Organizational  Processes? 

In  the  areas  of  communication  processes,  management 
attention,  decision-making,  and  financial  planning, 
communication  receives  the  most  positive  responses. 
Nearly  all  agree  that  both  formal  and  informal 
communication  processes  are  improved.  The  Award  Fee  is 
another  area  to  be  discussed,  and  contractor 
performance  must  be  communicated. 

Within  the  Air  Force  organization  there  is  an 
increased  participation  by  the  Contract  Administration 
Office,  and  there  is  more  management  attention  through 
briefings  prior  to  contract  award  and  through  the  fee 
determining  process. 

Even  though  there  is  increased  emphasis  on  the 
important  objectives,  there  appears  to  be  little 
noticeable  difference  in  contractor  management 
attention  above  the  middle  managers.  Government 
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managers  feel  the  amount  of  the  Award  Fee  is 
insignificant  to  a  large  government  contractor.  Other 
intangible  incentives,  such  as  getting  the  next 
contract,  may  influence  decision  making  and  management 
attention  more  than  the  Award  Fee. 

7.  What  are  the  effects  of  Award  Fee  on  Program  Outcomes? 

The  Award  Fee  will  not  solve  all  problems,  but  it 
did  not  hinder  the  program.  When  managed  properly,  it 
is  helpful,  yet  we  need  to  continue  with  other 
incentives.  By  using  the  Award  Fee,  important  points 
were  identifed  early,  and  a  review  structure  was 
created  to  evaluate  the  contractor's  response.  Even 
though  it  was  a  positive  motivator,  the  question 
becomes,  was  the  quality  of  the  product  any  better?  In 
the  final  analysis  of  ACES  II  there  was  probably  no 
effect  on  the  quality  of  the  product,  since  the  first 
300  ejection  seats  procured  by  the  follower  did 
experience  more  field  complaints  than  the  leader's 
seats.  However,  tnere  is  no  way  to  know  if  tnat  would 
have  been  any  different  without  the  Award  Fee. 

3.  Is  the  Award  Fee  used  in  Subcontracting? 

While  most  contractors  favor  being  on  the  receiving 
end  of  the  award  fee,  the  government  managers  had  no 
knowledge  of  the  prime  contractor  using  the  Award  Fee 
with  subcontractors. 
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9.  What  are  some  Strengths  and  Weaknesses  of  the  Award  Fee 
Method  of  Acquisition? 


The  primary  strength  from  the  .government  point  of 
view  is  the  value  of  a  management  tool  to  focus 
attention  in  specific  areas.  The  Award  Fee  has 
flexibility  over  other  incentive  contracts,  since  it  is 
not  tied  to  quantifiable  measures.  Meanwhile,  desires 
are  specifically  addressed  and  tied  to  the  cost  of  the 
contract.  Because  of  management's  focus  tnere  is 
increased  confidence  that  a  product  will  be  built  to 
specifications. 

In  contrast,  the  primary  weakness  seems  to  be  tnat 
the  contract  requirements  are  already  on  contract.  It 
is  difficult  to  assess  that  the  contractor  is 
performing  any  differently  as  a  result  of  the  award 
fee.  While  the  award  fee  is  subjectively  evaluated, 
objective  criteria  must  be  the  basis  for  the 
subjectively  determined  fee,  and  objective  criteria  are 
difficult  to  establish.  In  addition,  there  is  the 
possibility  that  the  cost  exceeds  the  benefits.  As 
part  of  the  cost,  the  additional  administrative 
workload  must  be  properly  managed  and  manned.  In  the 
front  end  of  the  program  there  is  a  considerable  amount 
of  work  involved  with  planning  and  structuring  the 
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Award  Fee  process. 


Finally,  the  fee  is  not  oig  enougn 


to  influence  a  large  government  contractor,  and 
motivated  people  are  necessary  to  implement  and  carry 
out  the  Award  Fee  process.  Since  it  is  difficult  to 
quantify,  che  Award  Fee  may  become  a  give  away  tnat  is 
influenced  by  a  political  decision. 

10.  What  are  the  Strengths  and  Weaxnesses  of  the  Award  Fee 

Method  for  Stimulating  3etter  Quality? 

Conveying  the  specific  incentive  to  improve  quality 
will  provide  management  attention  to  focus  on  quality 
consciousness.  The  Award  Fee  has  an  impact  on  future 
performance,  whereas  past  performance  is  evaluated  in 
source  selection.  The  potential  to  affect  future 
performance  requires  the  award  fee  to  be  used  early  in 
the  acquisition  cycle  to  build  a  better  quality  system 
early.  Also,  the  3ward  fee  could  be  used  as  late  as 
the  Production  Phase  in  a  sole  source  situation  wnere 
it  is  difficult  to  quantify  performance.  Since 

purchased  parts  are  important  to  the  quality  of  a 
system,  the  Award  Fee  should  motivate  the  contractor  to 
manage  his  subcontractors  and  vendors.  Additionally, 
the  Award  Fee  should  provide  an  incentive  to  the 
Contract  Administration  Office  to  track  and 
communicate  the  contractor's  performance. 
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Conversely,  the  Award  Fee  is  a  judgement  call  based 
on  experience  with  the  contractor  and  the  program.  We 
are  paying  for  extra  quality  when  there  is  no  true 
measure  of  quality.  Since  the  fee  is  subjective  we 
could  be  awarding  for  improvements  not  required  or 
paying  extra  for  no  improvemen ts .  Without  a  specific 
objective,  the  Award  Fee  should  not  be  used. 

11.  What  Quality  Criteria  should  be  used  in  the  Evaluation 
Process? 

The  majority  of  responses  preferred  evaluation 
criteria  based  on  objective  evidence.  For  quality  of 
conformance  there  are  many  objective  elements  based  on 
specifications  and  drawings.  In  addition,  yields  of 
processes,  inspections,  and  tests  are  methods  of  deter¬ 
mining  quality  of  manufactured  items.  Finally,  tne 
physical  configuration  audit,  functional  configuration 
audit,  and  first  article  will  demonstrate  conformance. 

Likewise  for  quality  of  design  there  are  many 
objective  elements  to  determine  the  fitness  for  use. 
Most  of  these  elements  are  related  to  reliability  and 
maintainability.  In  addition,  the  released  design  on 
engineering's  parts  and  tolerances  may  be  matched  to 
the  manufacturing  process.  while  some  thought  the  rate 
of  engineering  changes  would  be  a  good  criteria,  others 
believed  it  would  encourage  the  contractor  to  hide 
changes  until  after  the  award  period. 
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In  tne  area  of  management  of  quality  the  measurement 
criteria  become  subjective  when  determining  the  quality 
consciousness  of  the  contractor.  Some  elements  to  con¬ 
sider  include  the  status  of  the  chief  of  quality  within 
the  organization;  where  does  the  quality  reporting  stop 
within  the  chain  of  command;  management  of  subcontrac¬ 
tors  and  vendors;  the  cost  of  quality  which  includes 
scrap,  rework,  and  repair;  and  responsiveness  and 
resolution  of  corrective  action  within  the  plant  and  to 
field  complaints. 

Most  people  felt  uncomfortable  with  the  subjective 
criteria,  while  some  believe  that  objective  criteria 
may  present  a  facade  where  subjective  judgement  is  used 
to  obtain  the  objective  evidence.  Some  even  believe 
that  the  Award  Fee  is  inappropriate  when  there  is  objec¬ 
tive  criteria  available  for  an  incentive  contract. 

Overall,  the  effort  should  be  tailored  for  the 
specific  objectives  of  the  program.  The  emphasis  snould 
be  on  excellence  in  management  and  aggressiveness  in 
working  quality  issues. 

12.  What  influence  did  the  Award  Fee  have  on  the  design  of 
the  product? 

In  the  case  of  the  ACES  II,  the  Award  Fee  was  used 
in  a  leader-follower  program  after  the  design  of  tne 
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ejection  seat.  Approximately  2000  units  nad  been  Ouilt 
by  the  leader  prior  to  the  Award  Fee.  As  a  result, 
there  was  no  change  to  the  design  of  the  product  influ¬ 
enced  by  the  Award  Fee. 

For  the  Wild  Weasel  PUP,  the  Award  Fee  was  incor¬ 
porated  in  the  second  RFP  issued  for  the  development 
program.  The  terms  called  for  a  potential  Award  Fee  on 
the  later  production  contract.  Government  managers 
hoped  for  an  influence  on  the  design  by  providing 

advance  knowledge  of  the  Award  Fee.  Government  managers 
believe  it  is  too  early  in  the  program  to  determine  if 
any  influence  occurred  to  the  design. 

13.  What  problems  occurred  with  the  Quality  Criteria 
associated  with  the  Evaluation  Process? 

In  a  subjective  situation  quantifying  performance 
against  evaluation  criteria  is  difficult.  For  ACES  II 
the  leader  was  to  elevate  the  follower  to  become  a 
direct  competitor.  This  in  itself  poses  a  unique  pro¬ 

blem  in  motivating  a  contractor  to  give  away  part  of 
his  business.  In  addition,  an  ejection  seat  is  a 
crewmen's  last  means  of  escape  in  a  life  threatening 

situation.  As  a  result,  the  seat  is  quality  sensitive 

with  desired  high  reliability.  In  this  situation,  tne 
Award  Fee  was  based  on  how  well  the  leader  instructed. 
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Government  managers  felt  future  situations  should  include 
how  well  the  follower  learned.  The  goal  is  to  be  satis¬ 
fied  that  performance  objectives  are  met,  and  it  is 
difficult  to  distinguish  "over  and  above"  effort  from 
normal  effort.  In  the  case  of  Wild  Weasel,  the  evalua¬ 
tion  criteria  have  not  been  used,  however,  government 
managers  hope  for  a  positive  motivator  while  realizing 
that  perfection  is  seldom  achieved. 

While  quality  areas  may  be  redundant,  government 
managers  believe  it  is  better  to  be  redundant  than  to 
miss  something  significant,  especially  in  the  case  of  a 
critical  item,  which  could  cause  loss  of  a  life  or 
mission  failure.  Consequently,  in  designing  the  criteria 
there  was  no  effort  to  make  criteria  mutually  exclusive. 
Also,  because  of  the  inter-relationships,  it  is  necessary 
to  have  flexibility  in  the  evaluation  areas  and  process. 
Since  the  Award  Fee  is  "over  and  above"  the  normal  con¬ 
tract  profit,  it  should  be  a  positive  motivator. 

14.  What  feedback  does  the  contractor  receive  about  the 
product  in  the  field? 

The  Air  Force  has  reliability,  maintainability,  and 
availability  data  available  to  the  contractor  with  every 
program  having  different  ar rangements .  For  an  initial 
deployment,  it  may  be  necessary  to  have  contractor 
technical  representatives  in  the  field  to  examine  on  site 
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conditions  and  failures.  This  provides  prompt  informa¬ 
tion  to  the  contractor  and  supplements  the  formal  Air 
Force  system. 

In  the  case  of  ACES  II,  the  ejection  system  was 
already  designed  and  proven.  Reliability  and  maintain¬ 
ability  were  in  the  specification  with  no  new  require¬ 
ments.  Since  Wild  Weasel  PUP  is  ii  the  development 
stage,  reliability  and  maintainability  must  be  proven  in 
the  future. 

The  largest  problem  seems  to  be  providing  feedback 
on  system  performance  to  the  subcontractors,  althougn 
there  are  efforts  to  improve  this  communication  link. 


CHAPTER  4 


CONCLUSIONS  AND  RECOMMENDATIONS 

Before  any  conclusions  are  drawn  from  the  previous 
chapters,  it  must  be  remembered  that  the  two  cases  involved 
two  divisions  of  a  single  corporation,  which  has  been  a 
major  DoD  contractor.  In  addition  the  ACES  II  was  a  leader- 
follower  situation  where  the  Award  Fee  had  a  primary  goal  of 
orienting  a  follower  to  provide  a  quality  product  in  a 
reasonable  time  consistent  with  the  leader's  past  perfor¬ 
mance.  The  enhanced  product  quality  was  on  the  training 
provided  by  the  leader,  not  the  ejection  seat.  As  a  result 
of  the  training,  the  follower's  seat  should  have  been  just 
as  good  as  the  leader's  seat.  Also  in  the  case  of  the  Wild 
Weasel,  the  primary  goal  was  to  motivate  the  sole  source 
contractor  to  control  production  and  life  cycle  costs 
without  sacrificing  quality  of  the  product.  The  Wild  Weasel 
is  an  existing  aircraft,  the  Performance  Update  Program 
contract  was  awarded  to  the  sole  source  aircraft  contractor 
to  obtain  an  updated  electronics  package  for  the  aircraft. 
Hopefully,  the  contractor  will  achieve  the  design  and 
control  the  costs  without  degradation  to  the  existing 
aircraft's  reliability.  The  desired  enhanced  product 

quality  is  on  the  electronic  hardware.  In  each  situation, 
the  Air  Force  is  purchasing  management  of  the  desired  goal. 
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Conclusions 


Conclusion  One 

A  set  of  standard  quality  evaluation  areas  are  feasible 
for  use  with  all  award  fee  contracts,  but  specific  quality 
evaluation  criteria  should  be  unique  to  each  situation. 

Based  on  the  criteria  test  in  Chapter  2,  all  three 
research  questions  produced  some  common  quality  evaluation 
areas.  As  a  result  of  the  Award  Fee  plans  and  the  inter¬ 
views  of  government  and  contractor  personnel,  the  following 
quality  evaluation  areas  should  be  used  for  a  quality 
program: 

1.  Internal  quality  management 
a.  Managerial  activity 

(1)  Management  involvement. 

(2)  Objectives  set. 

(3)  Status  of  the  chief  of  quality. 

(4)  Internal  quality  discipline. 

(5)  Independence  of  the  quality  assurance 

department . 

(5)  Proper  manning  including  knowledge, 

experience,  and  employee  motivation. 

(7)  Trend  analysis  and  corrective  action. 
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b.  Technological  activity. 

(1)  Quality  of  design. 

(2)  Reliability. 

(3)  Producibility . 

(4)  Cost  of  quality. 

2.  External  quality  management 

a.  Supplier  management  and  control  of  purchase. 

b.  Nonconforming  material  and  corrective  action. 

Evaluation  criteria  must  be  thoughtfully  chosen  to  match 

the  desired  objectives.  Since  each  contract  has  different 
objectives,  standard  evaluation  criteria  for  each  evaluation 
area  may  not  be  applicable.  Also,  by  presenting  a  standard 
grading  criteria,  contractors  may  merely  comply  with  the 
published  criteria  instead  of  being  creative  towards  quality 
improvement.  The  primary  concern  is  managment  of  the  quality 
system.  How  does  management  plan  and  control  quality  inter¬ 
nal  to  the  organization  and  quality  external  to  the  organiza¬ 
tion?  The  decisions  and  decision  process  which  management 
uses  to  bring  a  positive  change  of  quality  should  influence 
the  Award  Fee.  While  Quality  of  Design  and  Quality  of  Con¬ 
formance  are  important,  these  areas  can  usually  be  evaluated 
objectively.  MIL-Q-9858A  contains  objective  criteria  to 
stress  compliance  to  the  contract.  However,  the  Award  Fee 
should  be  aimed  above  and  beyond  compliance  to  a  contract. 
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Excellence  in  management  and  aggressiveness  in  working 
quality  issues  are  determined  by  the  policies  and  objectives 
which  management  communicates  verbally  and  in  writing.  These 
policies  and  objectives  set  the  stage  for  quality  of  the 
product  and  quality  improvement. 

Conclusion  Two 

The  Award  Fee  is  a  potent  motivator  at  tne  program  level 
and  should  be  used  early  in  the  design  phase. 

A  competitive  company  will  strive  to  meet  its  goals, 
even  to  the  extent  that  it  may  later  lose  a  portion  of  its 
business.  If  nothing  else,  the  Award  Fee  forces  the  contrac¬ 
tor  to  think  of  alternatives  to  accomplishing  the  task.  3y 
winning  a  major  portion  of  the  Award  Fee,  there  is  recogni¬ 
tion  of  a  job  well  done,  which  promotes  esprit  de  corps 
among  the  program  manager  and  his  staff.  While  the  Award 
Fee  was  not  directly  filtered  down  to  the  workers  on  the 
program,  the  Program  Manager  was  evaluated  on  the  program's 
profitability  which  the  Award  Fee  directly  affects. 

Because  the  ACES  II  program  had  high  visibility  prior  to 
the  Award  Fee,  there  can  be  no  conclusions  drawn  about  upper 
management’s  involvement.  Most  individuals  in  both  programs 
believed  upper  management  displayed  the  same  involvement  as 
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a  "normal  contract".  However,  on  the  Air  Force  side,  tner e 
were  additional  briefings  to  upper  management  on  the  Award 
Fee  in  the  planning  stage.  Also  management  was  involved  in 
the  evaluation  stage.  As  a  result,  the  government  personnel 
extensively  planned  the  contract  and  evaluated  tne  contrac¬ 
tor.  In  the  case  of  ACES  II,  the  program  manager  initially 
recommended  the  Award  Fee  and  leader-follower  program  be 
refused.  Upper  management  made  a  decision  to  accept  tne 
contract,  it  was  not  based  solely  on  the  Award  Fee.  Since 
it  would  involve  future  competition  for  one  of  the  company’s 
product  lines,  the  size  and  diversity  of  the  company  snould 
have  influenced  this  decision  to  accept  the  leader-follower 
and  Award  Fee  program  with  the  contract. 

These  instances  verified  Hunt's  finding  that  the  Award 
Fee  is  a  potent  motivator,  yet  the  evaluation  period  must  be 
early  enough  to  affect  management.  To  acnieve  the  maximum 
benefit  from  motivation,  the  Award  Fee  should  be  used  early 
to  identify  important  goals  and  objectives.  In  addition, 
the  contractor  is  most  aware  of  tne  Award  Fee  and  tne  goals 
during  the  evaluation  period.  Meanwhile,  in  wild  .;e3sel  the 
Award  Fee  was  proclaimed  early  enough,  but  tne  contractor 
personnel  were  not  accountable  until  the  production  contract. 
When  this  happens,  tne  length  of  time  between  the  goal  for¬ 
mation  and  the  evaluation  period  minimizes  the  effect  of  the 
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Award  Fee  on  design  considerations.  Moreover,  in  ACES  II 
the  contractor  took  feedback  or  lack  of  feedback  from  tne 
first  evaluation  period  to  determine  how  to  proceed  in  the 
second  period.  The  percentage  of  the  Award  Fee  from  the 
first  period  provided  a  measure  of  customer  satisfaction  and 
influenced  the  performance  or  change  to  performance  for  tne 
second  evaluation  period. 

Conclusion  Three 

For  the  Award  Fee  to  influence  Quality,  Upper  Management 
must  stress  and  promote  Quality  Consciousness  throughout  tne 
company. 

With  management  providing  leadersnip  and  stressing 
■quality  consciousness ,  all  areas  of  the  company  become 
involved  with  the  goal  of  quality  enhancement  instead  of 
just  the  program  manager  and  his  staff.  The  more  people 
that  become  involved,  the  more  suggestions  that  will  oe 
generated . 

When  the  Award  Fee  limits  involvement  to  tne  program 
manager  and  his  staff,  the  quality  organization  is  outside 
of  the  enhancement  process  or  becomes  limited  to  tne  few 
quality  members  on  the  program  manager's  staff.  Any 
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suggestions  for  an  improved  quality  system  will  nave  to  go 
through  a  quality  organization  which  is  not  tasked  to  see* 
improvement.  In  both  ACES  II  and  Wild  Weasel,  the  companies 
exhibited  a  quality  consciousness  prior  to  the  Award  Fee. 
Two  factors  which  influenced  this  quality  consciousness  are 
the  (J.S.  trend  to  be  more  quality  conscious  because  of  the 
Japanese  quality,  and  the  product  as  a  life  support  system 
which  provides  crewmen  a  means  of  escape  from  a  life- 
threatening  situation. 

Summary 

The  Award  Fee  is  good  for  promoting  enhanced  product 
quality  or  any  objective.  3y  stressing  a  goal,  managers  are 
forced  to  consider  alternatives  which  may  improve  the  situ¬ 
ation.  Evaluation  criteria  must  be  thoughtfully  chosen  to 
match  the  objectives.  For  quality  enhancement,  standard 
evaluation  areas  may  be  used  and  should  focus  on  management's 
ability  and  attitude  toward  quality.  Finally,  it  is  impor¬ 
tant  to  have  upper  managment  leading  the  total  company  to  a 
state  of  quality  consciousness  and  creativity  towards  quality 
improvement . 

Recommendations 

In  the  Weapon  Acquisition  Process,  each  new  system  is 
unique  and  rarely  fits  into  a  predetermined  mold. 
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Consequently,  the  contracting  process  is  constantly 
changing  to  meet  the  specific  requirements  of  the  current 
situation.  Since  contracting  for  a  major  weapon  system  is 
an  iterative  process,  I  propose  a  number  of  recommendations 
based  on  the  ACES  II  and  Wild  Weasel  PUP  programs. 


Recommendation  One 

Quality  Evaluation  Areas  should  be  devoted  to  measuring 
management's  ability,  attitude,  and  leadership  towards 
quality  enhancement  such  as  those  evaluation  areas  listed 
in  Conclusion  One. 


Recommendation  Two 

To  promote  the  involvement  of  upper  level  management, 
quality  objectives  should  be  stated  and  periodic  reports  on 
progress  in  meeting  these  objectives  should  be  required 
during  the  evaluation  period. 


Recommendation  Three 

The  contractor  should  participate  in  the  selection  of 
the  quality  evaluation  areas  and  criteria  to  demonstrate  his 
understanding  of  the  objectives  of  the  Award  Fee. 
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Recommendation  Four 


More  study  is  needed  on  the  effects  of  the  Award  Fee  tor 
quality  enhancement,  especially  in  the  area  of  the  motiva¬ 
tional  value  of  the  size  of  the  Award  Fee  and  the  relation¬ 
ship  with  the  size  of  the  company. 

Recommendation  Five 

The  Quality  Award  Fee  should  be  tried  on  a  small  company 
that  does  not  have  a  large  product  line  and  that  has  nad  a 
history  of  quality  problems. 

Recommendation  Six 

The  evaluation  period  for  Quality  Award  Fees  should 
begin  during  the  early  stages  of  product  design  and  develop¬ 
ment. 

Recommenda tion  Seven 

A  study  should  be  conducted  to  establish  specific  evalu¬ 
ation  criteria  that  could  be  used  for  each  evaluation  area 
listed  in  Conclusion  One. 
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MIL-0- 9858A 
AMENDMENT  1 
7  August  1981 


MILITARY  SPECIFICATION 
QUALITY  PROGRAM  REQUIREMENTS 

This  amendment  forms  a  part  of  Military  Specification  MIL-Q-9858A, 
dated  16  December  1963,  and  is  approved  for  use  by  all  Departments 
and  Aqencies  of  the  Department  of  Defense. 

PAGE  2 

2.1,  lines  9  and  10:  Delete  "MIL-C-45662--Calibration  System  Requirements"  and 
substitute:  "MIL-Sro-45662--Cal ibration  Systems  Requirements". 

PAGE  4 

4.2,  line  15:  Delete  "MIL-C-45662  "and  substitute:  "MIL-STD-45662" . 


Custodians: 

Army  -  AR 
Navy  -  NM 
Air  Force  -  05 


Preparing  Activity: 
Air  Force  -  05 


MIL-Q-9858A 

16  DECEMBER  1963 

SUPERSEDING 
MIL-Q-9858 
9  APRIL  1959 


MILITARY  SPECIFICATION 

QUALITY  PROGRAM  REQUIREMENTS 

This  specification  hat  been  approved  by  the  Department  of  Defense  and  is  mandatory  for  use  by 


the  Departments  of  the  Army,  the  Navy,  the 

1.  SCOPE 

1.1  Applicability.  This  specification  shall 
apply  to  all  supplies  (including  equipments, 
sub-systems  and  systems)  or  services  when 
referenced  in  the  item  specification,  contract 
or  order. 

1.2  Contractual  Intent.  This  specification 
requires  the  establishment  of  a  quality  pro¬ 
gram  by  the  contractor  to  assure  compliance 
with  the  requirements  of  the  contract.  The 
program  and  procedures  used  to  implement 
this  specification  shall  be  developed  by  the 
contractor.  The  quality  program,  including 
procedures,  processes  and  product  shall  be 
documented  and  shall  be  subject  to  review 
by  the  Government  Representative.  The  qual¬ 
ity  program  is  subject  to  the  disapproval  of 
the  Government  Representative  whenever 
the  contractor’s  procedures  do  not  accom¬ 
plish  their  objectives.  The  Government,  at  its 
option,  may  furnish  written  notice  of  the 
acceptability  of  the  contractor’s  quality  pro¬ 
gram. 

1.3  Summary.  An  effective  and  economical 
quality  program,  planned  and  developed  in 
consonance  with  the  contractor’s  other  ad¬ 
ministrative  and  technical  programs,  is  re¬ 
quired  by  this  specification.  Design  of  the 
program  shall  be  based  upon  consideration  of 
the  technical  and  manufacturing  aspects  of 
production  and  related  engineering  design 
and  materials.  The  program  shall  assure 
adequate  quality  throughout  all  areas  of  con¬ 
tract  performance;  for  example,  design,  de¬ 
velopment,  fabrication,  processing,  assembly, 
inspection,  test,  maintenance,  packaging, 
shipp’ng,  storage  and  site  installation. 


Air  Fores  and  the  Defense  Supply  Agency. 

All  supplies  and  services  under  the  con¬ 
tract,  whether  manufactured  or  performed 
within  the  contractor’s  plant  or  at  any  other 
source,  shall  be  controlled  at  all  points  nec¬ 
essary  to  assure  conformance  to  contractual 
requirements.  The  program  shall  provide  for 
the  prevention  and  ready  detection  of  dis¬ 
crepancies  and  for  timely  and  positive  cor¬ 
rective  action.  The  contractor  shall  make 
objective  evidence  of  quality  conformance 
readily  available  to  the  Government  Repre¬ 
sentative.  Instructions  and  records  for  qual¬ 
ity  must  be  controlled. 

The  authority  and  responsibility  of  those 
in  charge  of  the  design,  production,  testing, 
and  inspection  of  quality  shall  be  clearly 
stated.  The  program  shall  facilitate  deter¬ 
minations  of  the  effects  of  quality  deficiencies 
and  quality  costs  on  price.  Facilities  and 
standards  such  ns  drawings,  engineering 
changes,  measuring  equipment  and  the  like 
which  are  necessary  for  the  creation  of  the 
required  quality  shall  be  effectively  managed. 
The  program  shall  include  an  effective  con¬ 
trol  of  purchased  materials  and  subcontract¬ 
ed  work.  Manufacturing,  fabrication  and 
assembly  work  conducted  within  the  contrac¬ 
tor’s  plant  shall  be  controlled  completely 
The  quality  program  shall  also  include  ef¬ 
fective  execution  of  responsibilities  shared 
jointly  with  the  Government  or  related  to 
Government  functions,  such  as  control  of 
Government  property  and  Government 
source  inspection. 

1.4  Relation  to  Other  Contract  Require¬ 
ments.  This  specification  and  any  procedure  or 
document  executed  in  implementation  there- 
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of,  shall  be  in  addition  to  and  not  in  deroga¬ 
tion  of  other  contract  requirements.  The 
quality  program  requirements  set  forth  in 
this  specification  shall  be  satisfied  in  addition 
to  all  detail  requirements  contained  in  the 
statement  of  work  or  in  other  parts  of  the 
contract.  The  contractor  is  responsible  for 
compliance  with  all  provisions  of  the  con¬ 
tract  and  for  furnishing  specified  supplies 
and  services  which  meet  all  the  requirements 
of  the  contract.  If  any  inconsistency  exists 
between  the  contract  schedule  or  its  general 
provisions  and  this  specification,  the  contract 
schedule  and  the  general  provisions  shall 
control.  The  contractor’s  quality  program 
shall  be  planned  and  used  in  a  manner  to 
support  reliability  effectively. 

1.5  Relation  to  MIL-I— 15208.  This  specifi¬ 
cation  contains  requirements  in  excess  of 
those  in  specification  MIL-I-45208,  Inspec¬ 
tion  System  Requirements,  inasmuch  as  total 
conformance  to  contract  requirements  is  ob¬ 
tained  best  by  controlling  work  operations, 
manufacturing  processes  as  well  as  inspec¬ 
tions  and  tests. 

2.  SUPERSEDING,  SUPPLEMENTA¬ 
TION  AND  ORDERING 

2.1  Applicable  Documents.  The  following 
documents  of  the  issue  in  effect  on  date  of 
the  solicitation  form  a  part  of  this  specifica¬ 
tion  to  the  extent  specified  herein. 

SPECIFICATIONS 

Military 

MIL-I—15208  — Inspection  System  Re¬ 
quirements 

MIL-C-45662  — Calibration  System 

Requirements 

2.2  Amendments  and  Revisions.  Whenever 
this  specification  is  amended  or  revised  sub¬ 
sequent  to  its  contractually  effective  date, 
the  contractor  may  follow  or  authorize  his 
subcontractors  to  follow  the  amended  or  re¬ 
vised  document  provided  no  increase  in  price 
or  fee  is  required.  The  contractor  shall  not 
be  required  to  follow  the  amended  or  revised 
document  except  as  a  change  in  contract.  If 


the  contractor  elects  to  follow  the  amended 
or  revised  document,  he  shall  notifv  the  Con¬ 
tracting  Officer  in  writing  of  this  election. 
When  the  contractor  elects  to  follow  the 
provisions  of  an  amendment  or  revision,  he 
must  follow  them  in  full. 

2.3  Ordering  Government  Documents.  Cop¬ 
ies  of  specifications,  standards  and  drawings 
required  by  contractors  in  connection  with 
specific  procurements  may  be  obtained. from 
the  procuring  agency,  or  as  otherwise  direct¬ 
ed  by  the  Contracting  Officer. 

3.  QUALITY  PROGRAM  MANAGEMENT 

3.1  Organization.  Effective  management 
for  quality  shall  be  clearly  prescribed  by  the 
contractor.  Personnel  performing  quality 
functions  shall  have  sufficient,  well-defined 
responsibility,  authority  and  the  organiza¬ 
tional  freedom  to  identify  and  evaluate  qual¬ 
ity  problems  and  to  initiate,  recommend  or 
provide  solutions.  Management  regularly 
shall  review  the  status  and  adequacy  of  the 
quality  program.  The  term  “quality  program 
requirements”  as  used  herein  identifies  the 
collective  requirements  of  this  specification. 
It  does  not  mean  that  the  fulfillment  of  the 
requirements  of  this  specification  is  the  re¬ 
sponsibility  of  any  single  contractor's  organ¬ 
ization,  function  or  person. 

3.2  Initial  Quality  Planning.  The  contrac¬ 
tor,  during  the  earliest  practical  phase  of 
contract  performance,  shall  conduct  a  com¬ 
plete  review  of  the  requirements  of  the  con¬ 
tract  to  identify  and  make  timely  provision 
for  the  special  controls,  processes,  test  equip¬ 
ments,  fixtures,  tooling  and  skills  required 
for  assuring  product  quality.  This  initial 
planning  will  recognize  the  need  and  provide 
for  research,  when  necessary,  to  update  in¬ 
spection  and  testing  techniques,  instrumenta- 
tiort  and  correlation  of  inspection  and  test 
results  with  manufacturing  methods  and 
processes.  This  planning  will  also  provide 
appropriate  review  and  action  to  assure  com¬ 
patibility  of  manufacturing,  inspection,  test¬ 
ing  and  documentation. 

3.3  Work  Instructions.  The  quality  pro¬ 
gram  shall  assure  that  all  work  affecting 
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quality  (including  such  things  as  pur¬ 
chasing,  handling,  machining,  assembling, 
fabricating,  processing,  inspection,  testing, 
modification,  installation,  and  any  other 
treatment  of  product,  facilities,  standards  or 
equipment  from  the  ordering  of  materials  to 
dispatch  of  shipments)  shall  be  prescribed 
in  clear  and  complete  documented  instruc¬ 
tions  of  a  type  appropriate  to  the  circum¬ 
stances.  Such  instructions  shall  provide  the 
criteria  for  performing  the  work  functions 
and  they  shall  be  compatible  with  acceptance 
criteria  for  workmanship.  The  instructions 
are  intended  also  to  serve  for  supervising, 
inspecting  and  managing  work.  The  prepara¬ 
tion  and  maintenance  of  and  compliance  with 
work  instructions  shall  be  monitored  as  a 
function  of  the  quality  program. 

3.4  Records.  The  contractor  shall  maintain 
and  use  any  records  or  data  essential  to  the 
economical  and  elTective  operation  of  his 
quality  program.  These  records  shall  bo 
available  for  review  by  the  Government  Rep¬ 
resentative  and  copies  of  individual  records 
shall  be  furnished  him  upon  request.  Records 
are  considered  one  of  the  principal  forms  of 
objective  evidence  of  quality.  The  quality 
program  shall  assure  that  records  are  com¬ 
plete  and  reliable.  Inspection  and  testing  rec¬ 
ords  shall,  as  a  minimum,  indicate  the  nature 
of  the  observations  together  with  the  num¬ 
ber  of  observations  made  and  the  number 
and  type  of  deficiencies  found.  Also,  records 
for  monitoring  work  performance  and  for 
inspection  and  testing  shall  indicate  the  ac¬ 
ceptability  of  work  or  products  and  the  ac¬ 
tion  taken  in  connection  with  deficiencies. 
The  quality  program  shall  provide  for  the 
analysis  and  use  of  records  as  a  basis  for 
management  action. 

3.5  Corrective  Action.  The  quality  program 
shall  detect  promptly  and  correct  assignable 
conditions  adverse  to  quality.  Design,  pur¬ 
chasing,  manufacturing,  testing  or  other 
operations  which  could  result  in  or  have  re¬ 
sulted  in  defective  supplies,  services,  facili¬ 
ties,  technical  data,  standards  or  other 
elements  of  contract  performance  which 
could  create  excessive  losses  or  costs  must 
be  identified  and  changed  as  a  result  of  the 


quality  program.  Corrective  action  will  ex¬ 
tend  to  the  performance  of  all  suppliers  and 
vendors  and  will  be  responsive  to  data  and 
product  forwarded  from  users.  Corrective 
action  shall  include  as  a  minimum: 

(a)  Analysis  of  data  and  examination  of 
product  scrapped  or  reworked  to  determine 
extent  and  causes; 

(b)  Analysis  of  trends  in  processes  or 
performance  of  work  to  prevent  noncon¬ 
forming  product;  and 

(c)  Introduction  of  required  improve¬ 
ments  and  corrections,  an  initial  review  of 
the  adequacy  of  such  measures  and  monitor¬ 
ing  of  the  effectiveness  of  corrective  action 
taken. 

3.6  Costs  Related  to  Quality.  The  contrac¬ 
tor  shall  maintain  and  use  quality  cost  data 
as  a  management  element  of  the  quality  pro¬ 
gram.  These  data  shall  serve  the  purpose  of 
identifying  the  cost  of  both  the  prevention 
and  correction  of  nonconforming  supplies  (e. 
g.,  labor  and  material  involved  in  material 
spoilage  caused  by  defective  work,  correc¬ 
tion  of  defective  work  and  for  quality  control 
exercised  by  the  contractor  at  subcontrac¬ 
tor's  or  vendor’s  facilities).  The  specific  qual¬ 
ity  cost  data  to  be  maintained  and  used  will 
be  determined  by  the  contractor.  Those  data 
shall,  on  request,  be  identified  ami  made  avail¬ 
able  for  “on  site”  review  by  the  Government 
Representative. 

I.  FACILITIES  AND  STANDARDS 

4.1  Drawings,  Documentation  and  Changes. 
A  procedure  shall  be  maintained  that  con¬ 
cerns  itself  with  the  adequacy,  the  complete¬ 
ness  and  the  currentness  of  drawings  and 
with  the  control  of  changes  in  design.  With 
respect  to  the  currentness  of  drawings  and 
changes,  the  contractor  shall  assure  that  re¬ 
quirements  for  the  eireetivity  point  of 
changes  are  met  and  that  obsolete  drawings 
and  change  requirements  are  removed  from 
all  points  of  issue  and  use.  Some  means  of 
recording  the  effective  points  shall  be  em¬ 
ployed  and  be  available  to  the  Government. 

With  respect  to  design  drawings  and  de¬ 
sign  specifications,  a  procedure  shad  be 
maintained  that  shall  provide  for  the  evalua- 
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tion  of  their  engineering  adequacy  and  an 
evaluation  of  the  adequacy  of  proposed 
changes.  The  evaluation  shall  encompass 
both  the  adequacy  in  relation  to  standard 
engineering  and  design  practices  and  the 
adequacy  with  respect  to  the  design  and  pur¬ 
pose  of  the  product  to  which  the  drawing  re¬ 
lates. 

With  respect  to  supplemental  speci¬ 
fications,  process  instructions,  production 
engineering  instructions,  industrial  engineer¬ 
ing  instructions  and  work  instructions  re¬ 
lating  to  a  particular  design,  the  contractor 
shall  be  responsible  for  a  review  of  their 
adequacy,  currentness  and  completeness.  The 
quality  program  must  provide  complete  cov¬ 
erage  of  all  information  necessary  to  produce 
an  article  in  complete  conformity  with  re¬ 
quirements  of  the  design. 

The  quality  program  shall  assure  that 
there  is  complete  compliance  with  contract 
requirements  for  proposing,  approving,  and 
effecting  of  engineering  changes.  The  quality 
program  shall  provide  for  monitoring  ef¬ 
fectively  compliance  v  ith  contractual  en¬ 
gineering  changes  requiring  approval  by 
Government  design  authority.  The  quality 
program  shall  provide  for  monitoring  effec¬ 
tively  the  drawing  changes  of  lesser  impor¬ 
tance  not  requiring  approval  by  Government 
design  authorities. 

Delivery  of  correct  drawings  and  change 
information  to  the  Government  in  connection 
with  data  acquisition  shall  be  an  integral  part 
of  the  quality  program.  This  includes  full 
compliance  with  contract  requirements  con¬ 
cerning  rights  and  data  both  proprietary  and 
other.  The  quality  program’s  responsibility 
for  drawings  and  changes  extend  to  the  draw¬ 
ings  and  changes  provided  by  the  subcontrac¬ 
tors  and  vendors  for  the  contract. 

4.2  Measuring  and  Testing  Equipment. 
The  contractor  shall  provide  and  maintain 
gages  and  other  measuring  and  testing  de¬ 
vices  necessary  to  assure  that  supplies  con¬ 
form  to  technical  requirements.  These 
devices  shall  be  calibrated  against  certified 
measurement  standards  which  have  known 
valid  relationships  to  national  standards  at 


established  periods  to  assure  continued  ac¬ 
curacy.  The  objective  is  to  assure  that  in¬ 
spection  and  test  equipment  is  adjusted, 
replaced  or  repaired  before  it  becomes  inac¬ 
curate.  The  calibration  of  measuring  and 
testing  equipment  shall  be  in  conformity 
with  military  specification  MIL-C— 15662.  In 
addition,  the  contractor  shall  insure  the  use 
of  only  such  subcontractor  and  vendor 
sources  that  depend  upon  calibration  systems 
which  effectively  control  the  accuracy  of 
measuring  and  testing  equipment. 

1.15  Production  Tooling  Used  as  Media  of 
Inspection.  When  production  jigs,  fixtures, 
tooling  masters,  templates,  patterns  and 
such  other  devices  arc  used  as  media  of  in¬ 
spection,  they  shall  be  proved  for  accuracy 
prior  to  release  for  use.  These  devices  shall 
be  proved  again  for  accuracy  at  intervals 
formally  established  in  a  manner  to  cause 
their  timely  adjustment,  replacement  or  re¬ 
pair  prior  to  becoming  inaccurate. 

1.4  Use  of  Contractor’s  Inspection  Equip¬ 
ment.  The  contractor’s  gages,  measuring  and 
testing  devices  shall  be  made  available  for 
use  by  the  Government  when  require i  to 
determine  conformance  with  contract  re¬ 
quirements.  If  conditions  warrant,  contrac¬ 
tor’s  personnel  shall  be  made  available  for 
operation  of  such  devices  and  for  verification 
of  their  accuracy  and  condition. 

4.5  Advanced  Metrology  Requirements. 
The  quality  program  shall  include  timely 
identification  and  report  to  the  Contracting 
Officer  of  any  precision  measurement  need 
exceeding  the  known  state  of  the  art. 

5.  CONTROL  OF  PURCHASES 

5.1  Responsibility.  The  contractor  is  re¬ 
sponsible  for  assuring  that  all  supplies  and 
services  procured  from  his  suppliers  (sub¬ 
contractors  and  vendors)  conform  to  the 
contract  requirements.  The  selection  of 
sources  and  the  nature  and  extent  of  control 
exercised  by  the  contractor  shall  be  depend¬ 
ent  upon  the  type  of  supplies,  his  supplier’s 
demonstrated  capability  to  perform,  and  the 
quality  evidence  made  available.  To  assure 
an  adequate  and  economical  control  of  such 
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material,  the  contractor  shall  utilize  to  the 
fullest  extent  objectives  evidence  of  quality 
furnished  by  his  suppliers.  When  the  Gov¬ 
ernment  elects  to  perform  inspection  at  a 
supplier’s  plant,  such  inspection  shall  not  be 
used  by  contractors  as  evidence  cf  effective 
control  of  quality  by  such  suppliers.  The  in¬ 
clusion  of  a  product  on  the  Qualified  Prod¬ 
ucts  List  only  signifies  that  at  one  time  the 
manufacturer  made  a  product  which  met 
specification  requirements.  It  does  not  relieve 
the  contractor  of  his  responsibility  for  furn¬ 
ishing  supplies  that  meet  all  specification  re¬ 
quirements  or  for  the  performance  of 
specified  inspections  and  tests  for  such  ma¬ 
terial.  The  effectiveness  and  integrity  of  the 
control  of  quality  by  his  suppliers  shall  be 
assessed  and  reviewed  by  the  contractor  at 
intervals  consistent  with  the  complexity  and 
quantity  of  product.  Inspection  of  products 
upon  delivery  to  the  contractor  shall  be  used 
for  assessment  and  revie%v  to  the  extent  nec¬ 
essary  for  adequate  assurance  of  quality. 
Test  reports,  inspection  records,  certificates 
and  other  suitable  evidence  relating  to  the 
supplier’s  control  of  quality  should  be  used  in 
the  contractor’s  assessment  and  review.  The 
contractor’s  responsibility  for  the  control  of 
purchases  includes  the  establishment  of  a 
procedure  for  (1)  the  selection  of  qualified 
suppliers,  (2)  the  transmission  of  applica¬ 
ble  design  and  quality  requirements  in  the 
Government  contracts  and  associated  tech¬ 
nical  requirements,  (3)  the  evaluation  of  the 
adequacy  of  procured  items,  and  (4)  effec¬ 
tive  provisions  for  early  information  feed¬ 
back  and  correction  of  nonconformances. 

5.2  Purchasing  Data.  The  contractor’s 
quality  program  shall  not  be  acceptable  to 
the  Government  unless  the  contractor  re¬ 
quires  of  his  subcontractors  a  quality  effort 
achieving  control  of  the  quality  of  the  serv¬ 
ices  and  supplies  which  they  provide.  The 
contractor  shall  assure  that  all  applicable  re¬ 
quirements  are  properly  included  or  refer¬ 
enced  in  all  purchase  orders  for  products 
ultimately  to  apply  on  a  Government  con¬ 
tract.  The  purchase  order  shall  contain  a 
complete  description  of  the  supplies  ordered 
including,  by  statement  or  reference,  all 


applicable  requirements  for  manufacturing, 
inspecting,  testing,  packaging,  and  any  re¬ 
quirements  for  Government  or  contractor 
inspections,  qualification  or  approvals.  Tech¬ 
nical  requirements  of  the  following  nature 
must  be  included  by  statement  or  reference 
as  a  part  of  the  required  clear  description : 
all  pertinent  drawings,  engineering  change 
orders,  specifications  (including  inspection 
system  or  quality  program  requirements), 
reliability,  safety,  weight,  or  other  special 
requirements,  unusual  test  or  inspection  pro¬ 
cedures  or  equipment  and  any  special  revi¬ 
sion  or  model  identification.  The  description 
of  products  ordered  shall  include  a  require¬ 
ment  for  contractor  inspection  at  the  sub¬ 
contractor  or  vendor  source  when  such  action 
is  necessary  to  assure  that  the  contractor’s 
quality  program  effectively  implements  the 
contractor's  responsibility  for  complete  as¬ 
surance  of  product  quality.  Requirements 
shall  be  included  for  chemical  and  physical 
testing  and  recording  in  connection  with  the 
purchase  of  raw  materials  by  his  suppliers. 
The  purchase  orders  must  also  contain  a  re¬ 
quirement  for  such  suppliers  to  notify  and 
obtain  approval  from  the  contractor  of 
changes  in  design  of  the  products.  Necessary 
instructions  should  be  provided  when  provi¬ 
sion  is  made  for  direct  shipment  from  the 
subcontractor  to  Government  activities. 

6.  MANUFACTURING  CONTROL 

6.1  Materials  and  Materials  Control.  Sup¬ 
plier’s  materials  and  products  shall  be  sub¬ 
jected  to  inspection  upon  receipt  to  the 
extent  necessary  to  assure  conformance  to 
technical  requirements.  Receiving  inspection 
may  be  adjusted  upon  the  basis  of  the  quality 
assurance  program  exercised  by  suppliers. 
Evidence  of  the  suppliers’  satisfactory  con¬ 
trol  of  quality  may  be  used  to  adjust  the 
amount  and  kind  of  receiving  inspection. 

The  quality  program  shall  assure  that  raw 
materials  to  be  used  in  fabrication  or  proc¬ 
essing  of  products  conform  to  the  applicable 
physical,  chemical,  and  other  technical  re¬ 
quirements.  Laboratory  testing  shall  be 
employed  as  necessary.  Suppliers  shall  be  re¬ 
quired  by  the  contractor’s  quality  program 
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to  exercise  equivalent  control  of  the  raw  ma¬ 
terials  utilized  in  the  production  of  the  parts 
and  items  which  they  supply  to  the  con¬ 
tractor.  Raw  material  awaiting  testing  must 
be  separately  identified  or  segregated  from 
already  tested  and  approved  material  but 
can  be  released  for  initial  production,  pro¬ 
viding  that  identification  and  control  is 
maintained.  Material  tested  and  approved 
must  be  kept  identified  until  such  time  as  its 
identity  is  necessarily  obliterated  by  process¬ 
ing.  Controls  will  be  established  to  prevent 
the  inadvertent  use  of  material  failing  to 
pass  tests. 

6.2  Production  Processing  and  Fabrication. 
The  contractor’s  quality  program  must  as¬ 
sure  that  all  machining,  wiring,  batching, 
shaping  and  all  basic  production  operations  of 
any  type  together  with  all  processing  and 
fabricating  of  any  type  is  accomplished  under 
controlled  conditions.  Controlled  conditions 
include  documented  work  instructions,  ade¬ 
quate  production  equipment,  and  any  special 
working  environment.  Documented  work  in¬ 
structions  are  considered  to  be  the  criteria 
for  much  of  the  production,  processing  and 
fabrication  work.  These  instructions  are  the 
criteria  for  acceptable  or  unacceptable  “work¬ 
manship”.  The  quality  program  will  effec¬ 
tively  monitor  the  issuance  of  and  compliance 
with  all  of  these  work  instructions. 

Physical  examination,  measurement  or 
tests  of  the  material  or  products  processed 
is  necessary  for  each  work  operation  and 
must  also  be  conducted  under  controlled  con¬ 
ditions.  If  physical  inspection  of  processed 
material  is  impossible  or  disadvantageous, 
indirect  control  by  monitoring  processing 
methods,  equipment  and  personnel  shall  be 
provided.  Both  physical  inspection  and  proc¬ 
ess  monitoring  shall  be  provided  when  con¬ 
trol  is  inadequate  without  both,  or  when 
contract  or  specification  requires  both. 

Inspection  and  monitoring  of  processed 
material  or  products  shall  be  accomplished  in 
any  suitable  systematic  manner  selected  by 
the  contractor.  Methods  of  inspection  and 
monitoring  shall  be  corrected  any  time  their 
unsuitability  with  reasonable  evidence  is 
demonstrated.  Adherence  to  selected  methods 


for  inspection  and  monitoring  shall  be  com¬ 
plete  and  continuous.  Corrective  measures 
shall  be  tak*n  when  noncompliance  occurs. 

Inspection  by  machine  operators,  automat¬ 
ed  inspection  gages,  moving  line  or  lot  sam¬ 
pling,  setup  or  first  piece  approval,  produc¬ 
tion  line  inspection  station,  inspection  or  test 
department,  roving  inspectors  —  any  other 
type  of  inspection  —  shall  be  employed  in 
any  combination  desired  by  the  contractor 
which  will  adequately  and  efficiently  protect 
product  quality  and  the  integrity  of  process¬ 
ing. 

Criteria  for  approval  and  rejection  shall 
be  provided  for  all  inspection  of  product  and 
monitoring  of  methods,  equipment,  and  per¬ 
sonnel.  Means  for  identifying  approved  and 
rejected  product  shall  be  provided. 

Certain  chemical,  metallurgical,  biological, 
sonic,  electronic,  and  radiological  processes 
are  of  so  complex  and  specialized  a  nature 
that  much  more  than  the  ordinary  detailing 
of  work  documentation  is  required.  In  ef¬ 
fect,  such  processing  may  require  an  entire 
work  specification  as  contrasted  with  the 
normal  work  operation  instructions  estab¬ 
lished  in  normal  plant-wide  standard  produc¬ 
tion  control  issuances  such  as  job  operation 
routing  books  and  the  like.  For  these  special 
processes,  the  contractors'  quality  program 
shall  assure  that  the  process  control  pro¬ 
cedures  or  specifications  are  adequate  and 
that  processing  environments  and  the  certify¬ 
ing,  inspection,  authorization  and  monitoring 
of  such  processes  to  the  special  degree  neces¬ 
sary  for  these  ultraprecise  and  supcr-oomplex 
work  functions  are  provided. 

6.3  Completed  Item  Inspection  and  Test¬ 
ing.  The  quality  program  shall  assure  that 
there  is  a  system  for  final  inspection  and  test 
of  completed  products.  Such  testing  shall 
provide  a  measure  of  the  overall  quality  of 
the  completed  product  and  shall  be  per¬ 
formed  so  that  it  simulates,  to  a  sufficient 
degree,  product  end  use  and  functioning. 
Such  simulation  frequently  involves  appro¬ 
priate  life  and  endurance  tests  and  qualifi¬ 
cation  testing.  Final  inspection  and  testing 
shall  provide  for  reporting  to  designers  any 
unusual  difficulties,  deficiencies  or  question- 


MIL-Q-9858A 


able  conditions.  When  modifications,  repairs 
or  replacements  are  required  after  final  in¬ 
spection  or  testing,  there  shall  be  reinspec¬ 
tion  and  retesting  of  any  characteristics 
affected. 

6.4  Handling,  Storage  and  Delivery.  The 
quality  program  shall  provide  for  adequate 
work  and  inspection  instructions  for  handl¬ 
ing,  storage,  preservation,  packaging,  and 
shipping  to  protect  the  quality  of  products 
and  prevent  damage,  loss,  deterioration,  de¬ 
gradation,  or  substitution  of  products.  With 
respect  to  handling,  the  quality  program 
shall  require  and  monitor  the  use  of  proce¬ 
dures  to  prevent  handling  damage  to  articles. 
Handling  procedures  of  this  type  include  the 
use  of  special  crates,  boxes,  containers,  trans¬ 
portation  vehicles  and  any  other  facilities 
for  materials  handling.  Means  shall  be  pro¬ 
vided  for  any  necessary  protection  against 
deterioration  or  damage  to  products  in  stor¬ 
age.  Periodic  inspection  for  the  prevention 
and  results  of  such  deterioration  or  damage 
shall  be  provided.  Products  subject  to  deteri¬ 
oration  or  corrosion  during  fabrication  or 
interim  storage  shall  be  cleaned  and  pre¬ 
served  by  methods  which  will  protect  against 
such  deterioration  or  corrosion.  When  nec¬ 
essary,  packaging  designing  and  packaging 
shall  include  means  for  accommodating  and 
maintaining  critical  environments  within 
packages,  e.g.,  moisture  content  levels,  gas 
pressures.  The  quality  program  shall  assure 
that  when  such  packaging  environments 
must  be  maintained,  packages  are  labeled  to 
indicate  this  condition.  The  quality  program 
shall  monitor  shipping  work  to  assure  that 
products  shipped  are  accompanied  with  re¬ 
quired  shipping  and  technical  documents  and 
that  compliance  with  Interstate  Commerce 
Commission  rules  and  other  applicable  ship¬ 
ping  regulations  is  effected  to  assure  safe  ar¬ 
rival  and  identification  at  destination.  In 
compliance  with  contractual  requirements, 
the  quality  program  shall  include  monitoring 
provisions  for  protection  of  the  quality  of 
products  during  transit. 

6.5  Nonconforming  Material.  The  contrac¬ 
tor  shall  establish  and  maintain  an  effective 


and  positive  system  for  controlling  noncon¬ 
forming  material,  including  procedures  for 
its  identification,  segregation,  and  disposi¬ 
tion.  Repair  or  rework  of  nonconforming 
material  shall  be  in  accordance  with  docu¬ 
mented  procedures  acceptable  to  the  Govern¬ 
ment.  The  acceptance  of  nonconforming 
supplies  is  a  prerogative  of  and  shall  be  as 
prescribed  by  the  Government  and  may 
involve  a  monetary  adjustment.  All  noncon¬ 
forming  supplies  shall  be  positively  identified 
to  prevent  unauthorized  use,  shipment  and 
intermingling  with  conforming  supplies. 
Holding  areas  or  procedures  mutually  agree¬ 
able  to  the  contractor  and  the  Government 
Representative  shall  be  provided  by  the  con¬ 
tractor.  The  contractor  shall  make  known 
to  the  Government  upon  request  the  data 
associated  with  the  costs  and  losses  in  con¬ 
nection  with  scrap  and  with  rework  neces¬ 
sary  to  reprocess  nonconforming  material  to 
make  it  conform  completely. 

6.6  Statistical  Quality  Control  and  Analy¬ 
sis.  In  addition  to  statistical  methods  required 
by  the  contract,  statistical  planning,  analysis, 
tests  and  quality  control  procedures  may  be 
utilized  whenever  such  procedures  are  suit¬ 
able  to  maintain  the  required  control  of 
quality.  Sampling  plans  may  be  used  when 
tests  are  destructive,  or  when  the  records, 
inherent  characteristics  of  the  product  or 
the  noncritical  application  of  the  product, 
indicate  that  a  reduction  in  inspection  or 
testing  can  be  achieved  without  jeopardizing 
quality.  The  contractor  may  employ  sampling 
inspection  in  accordance  with  applicable  mil¬ 
itary  standards  and  sampling  plans  (e.g., 
from  MIL-STD-105,  MIL-STD-414,  or 
Handbooks  H  106,  107  and  108).  If  the  con¬ 
tractor  uses  other  sampling  plans,  they  shall 
be  subject  to  review  by  the  cognizant  Gov¬ 
ernment  Representative.  Any  sampling  plan 
used  shall  provide  valid  confidence  and  qual¬ 
ity  levels. 

6.7  Indication  of  Inspection  Status.  The 
contractor  shall  maintain  a  positive  system 
for  identifying  the  inspection  status  of  prod¬ 
ucts.  Identification  may  be  accomplished  by 
means  of  stamps,  tags,  routing  cards,  move 
tickets,  tote  box  cards  or  other  normal  con- 
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trol  devices.  Such  controls  shall  be  of  a  de¬ 
sign  distinctly  different  from  Government 
inspection  identification. 

7.  COORDINATED  GOVERNMENT/ 
CONTRACTOR  ACTIONS 

7.1  Government  Inspection  at  Subcontrac¬ 
tor  or  Vendor  Facilities.  The  Government  re¬ 
serves  the  right  to  inspect  at  source  supplies 
or  services  not  manufactured  or  performed 
with  the  contractor’s  facility.  Government 
inspection  shall  not  constitute  acceptance; 
nor  shall  it  in  any  way  replace  contractor 
inspection  or  otherwise  relieve  the  contrac¬ 
tor  of  his  responsibility  to  furnish  an  accept¬ 
able  end  item.  The  purpose  of  this  inspection 
is  to  assist  the  Government  Representative 
at  the  contractor’s  facility  to  determine  the 
conformance  of  supplies  or  services  with 
contract  requirements.  Such  inspection  can 
only  be  requested  by  or  under  authorization 
of  the  Government  Representative.  When 
Government  inspection  is  required,  the  con¬ 
tractor  shall  add  to  his  purchasing  document 
the  following  statement: 

“Government  inspection  is  required 
prior  to  shipment  from  your  plant. 
Upon  receipt  of  this  order,  prompt¬ 
ly  notify  the  Government  Represen¬ 
tative  who  normally  services  your 
plant  so  that  appropriate  planning 
for  Government  inspection  can  be 
accomplished.” 

When,  under  authorization  of  the  Govern¬ 
ment  Representative,  copies  of  the  purchas¬ 
ing  document  are  to  be  furnished  directly 
by  the  subcontractor  or  vendor  to  the  Gov¬ 
ernment  Representative  at  his  facility  rather 
than  through  Government  channels,  the  con¬ 
tractor  shall  add  to  his  purchasing  document 
a  statement  substantially  as  follows: 

“On  receipt  of  this  order,  promptly 
furnish  a  copy  to  the  Government 
Representative  who  normally  serv¬ 
ices  your  plant,  or,  if  none,  to  the 
nearest  Army,  Navy,  Air  Force,  or 
Defense  Supply  Agency  inspection 
office.  In  the  event  the  representa¬ 
tive  or  office  cannot  be  located,  our 
purchasing  agent  should  be  notified 
immediately.” 


All  documents  and  referenced  data  for  pur¬ 
chases  applying  to  a  Government  contract 
shall  be  available  for  review  by  the  Govern¬ 
ment  Representative  to  determine  compli¬ 
ance  with  the  requirements  for  the  control 
of  such  purchases.  Copies  of  purchasing  doc¬ 
uments  required  for  Government  purposes 
shall  be  furnished  in  accordance  with  the  in¬ 
structions  of  the  Government  Representa¬ 
tive.  The  contractor  shall  make  available  to 
the  Government  Representative  reports  of 
any  nonconformance  found  on  Government 
source  inspected  supplies  ar.d  shall  (when  re¬ 
quested)  require  the  supplier  to  coordinate 
with  his  Government  Representative  on  cor¬ 
rective  action. 

7.2  Government  Property. 

7.2.1  Government-furnished  Material. 
When  material  is  furnished  by  the  Govern¬ 
ment,  the  contractor’s  procedures  shall  in¬ 
clude  at  least  the  following: 

(a)  Examination  upon  receipt,  consistent 
with  practicability  to  detect  damage  in  tran¬ 
sit; 

(b)  Inspection  for  completeness  and  prop¬ 
er  type; 

(c)  Periodic  inspection  and  precautions 
to  assure  adequate  storage  conditions  and  to 
guard  against  damage  from  handling  and 
deterioration  during  storage; 

(d)  Functional  testing,  either  prior  to  or 
after  installation,  or  both,  as  required  by 
contract  to  determine  satisfactory  operation; 

(e)  Identification  and  protection  from  im¬ 
proper  use  or  disposition;  and 

(f)  Verification  of  quantity. 

7.2.2  Damaged  Government-furnished  Ma¬ 
terial.  The  contractor  shall  report  to  the 
Government  Representative  any  Govern¬ 
ment-furnished  material  found  damaged, 
malfunctioning,  or  otherwise  unsuitable  for 
use.  In  the  event  of  damage  or  malfunction¬ 
ing  during  or  after  installation,  the  contrac¬ 
tor  shall  determine  and  record  probable 
cause  and  necessity  for  withholding  material 
from  use. 

7.2.3  Bailed  Property.  The  contractor  shall, 
as  required  by  the  terms  of  the  Pailment 
Agreement,  establish  procedures  for  the  ade- 
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quate  storage,  maintenance  and  inspection  of 
bailed  Government  property.  Records  of  all 
inspections  and  maintenance  performed  on 
bailed  property  shall  be  maintained.  These 
procedures  and  records  shall  be  subject  to 
review  by  the  Government  Representative. 

8.  NOTES 

(The  following  information  is  provided 
solely  for  guidance  in  using  this  specification. 
It  has  no  contractual  significance.) 

8.1  Intended  Use.  This  specification  will 
apply  to  complex  supplies,  components,  equip¬ 
ments  and  systems  for  which  the  require¬ 
ments  of  MIL-I— 15208  are  inadequate  to  pro¬ 
vide  needed  quality  assurance.  In  such  cases, 
total  conformance  to  contract  requirements 
cannot  be  obtained  effectively  and  economic¬ 
ally  solely  by  controlling  inspection  and  test¬ 
ing.  Therefore,  it  is  essential  to  control  work 
operations  and  manufacturing  processes  as 

Custodians: 

Army — Munitions  Command 
Navy — Office  of  Naval  Material 
Air  Force — Hq  L'SAF 
DSA — Hq  DSA 


well  as  inspections  and  tests.  The  purpose  of 
this  control  is  not  only  to  assure  that  partic¬ 
ular  units  of  hardware  conform  to  contrac¬ 
tual  requirements,  but  also  to  assure  interface 
compatibility  among  these  units  of  hardware 
when  they  collectively  comprise  major  equip¬ 
ments,  sub-systems  and  systems. 

8.2  Exemptions.  This  specification  will  not 
be  applicable  to  types  of  supplies  for  which 
MIL-I-45208  applies.  The  following  do  not 
normally  require  the  application  of  this 
specification: 

(a)  Personal  services,  and 

(b)  Research  and  development  studies 

of  a  theoretical  nature  which  do 
not  require  fabrication  of  articles. 

8.3  Order  Data.  Procurement  documents 
should  specify  the  title,  number  and  date  of 
this  specification. 

Preparing  Activity: 

Air  Forte — Hq  US AF 
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Questions  for  Contractors  and  Government  Managers 


1.  In  the  case  of  the  _  Program,  describe  any 

effects  pertaining  to  organizational  and  staffing  patterns, 
which  resulted  from  the  award  fee  feature  in  the  contract. 

a.  How,  for  instance,  did  it  affect  the  way  you 
organized  or  staffed  your  program  office? 

b.  How  did  it  affect  the  nature  of  your  relations  with 
other  parts  of  your  company? 

c.  How  did  it  affect  the  nature  of  your  relations  with 
the  government  program  office? 

d.  ..with  other  government  offices? 

e.  Are  there  any  differences  in  the  way  your  program 
office,  or  its  components  operateas  a  resslt  of  the 
award  fee? 

f.  Will  you  develop  any  award  fee-oriented  management 
methods--e  .g .  ,  "gaming"  of  award  fee  evaluations,  bonus 
systems,  "intelligence"  gathering? 

g.  Do  you  expect  to  find  award  fee  administration  any 
more  or  less  costly  than  other  forms  of  contract? 

2.  What  information  w3s  given  to  contractors  about  the 
government  award  fee  plan? 

a.  What  were  the  performance  factors? 

b.  What  is  the  evaluation  process? 

c.  Who  would  be  evaluating? 

d.  When  would  evaluations  be  held? 


3.  What  input  do  contractors  contribute  to  Award  Fee 
evaluations? 

4.  What  feedback  is  given  to  the  Contractors  on  tne  Award 
Fee  evaluations? 

5.  What  are  the  Award  Fee  Objectives? 

6.  How  does  the  Award  Fee  affect  organizational 
processes?  Will  there  be  special  effects  on: 

a.  Communication  processes,  especially  between  you  and 
government  program  offices  and  personnel? 

b.  Management  attention-- the  things  you  notice,  the 
people  who  notice  them,  and  the  setting  of  priorities 
for  activities? 

c.  Decision  making  —  how  you  made  decisions,  or  who 
participated  in  them? 

d.  Financial  planning  at  program  and  at  corporate 
levels? 

7.  Overall,  do  you  believe  the  award  fee  will  help  or  harm 
the  program? 


3.  Is  the  Award  Fee  method  used  in  subcontracting? 


9.  As  you  view 

it,  what 

are 

the  general 

strengths 

and 

weaknesses  of  the 

Award  Fee 

method  of  acquisition? 

10.  As  you  view 

it,  what 

are 

the  general 

strengths 

and 

weaknesses  of  the  Award 

Fee 

method  of 

acquisition 

to 

stimulate  better  quality? 
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a.  In  regards  to  stimulating  better  quality,  new  would 
you  compare  the  Award  Fee  method  with  other  "incentive" 
contracting  methods? 

b.  In  regards  to  stimulating  better  quality,  have  you 
any  particular  views  on  when  Award  Fee  should  oe  used? 
should  not  be  used? 

11.  In  regards  to  your  program,  what  quality  criteria 
would  you  use  to  evaluate 

a.  The  product? 

b.  For  quality  of  design? 

c.  For  quality  of  conformance? 

d.  For  management  of  quality? 

e.  Objectively?  Subjectively? 

12.  Has  the  contractor  aware  of  the  Award  Fee  during  tne 
design  of  the  product?  How  did  this  influence  the  design 
of  the  product? 

13.  what  problems  do  you  expect  to  encounter  with  tne 
quality  criteria  used  by  the  government?  Do  you  expect 
redundancies? 

14.  what  feedback  do  you  receive  from  the  Air  Force 

a.  On  maintainability,  availability,  and  reliability? 

b.  On  a  regular  basis?  On  a  crisis  basis? 
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ACES  II  Evaluation  Criteria 


1. 

Procedures  Evaluation. 

2. 

Timel i ness/ Adequacy 

of  Corrective  Ac 

tion . 

3. 

Work  Instructions. 

4. 

Trend  Analysis  and 

Internal  Cost  of 

Qual ity . 

5. 

Calibration  System. 

5. 

Schedule  Completion 

( QA  aspects) . 

7. 

Major  Deficiencies. 

8. 

Cost  of  Quality. 

9. 

Supplier  Rating  and 

Selection . 

10. 

Quality  Planning. 

11. 

Source  Inspection. 

12.  Manufacturing/Tooling  Inspection. 

Wild  Weasel  PUP  Evaluation  Criteria 

Quality/Producibili ty 

1.  Adherence  to  Management  System  Requirements. 

a.  Quality  Program  per  MIL-Q-985SA. 

b.  Control  of  Suppliers  per  MIL-STD-1535A. 

c.  Corrective  Action  and  Disposition  of 
conforming  Material  per  MIL-5TD-1 520B . 


Non- 
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2.  Quality  of  Design  evidenced  by: 

a.  Identifying  critical  product  cnaracter istics  to 

fitness  of  use,  controls  are  established  for 
production/inspection/test,  and  design 

tolerances  are  compatible  with  manufacturing 
process  variability. 

b.  Number  of  ECPs  required  to  correct  design  or 

producibility  deficiencies  or  to  accomodate 
inability  to  meet  specific  performance 
requirements . 

c.  Incidents  requiring  TCTO  action  for  field 
inspection,  modification,  retrofit,  or 
replacement . 

d.  Early  and  successful  demonstration  of  inter¬ 

changeability  and  replaceabil i ty . 

3.  Effectiveness  of  contractor's  in-process  preven¬ 
tive  quality  control  activity  as  reflected  in  data: 

a.  In-process  failures  and  removals  of  purchased 

parts  and  components. 

b.  Rejection  and  return-to-vendor  trends  at 
receiving  inspection. 

c.  Parts  control  and  parts  screening  for  semi¬ 

conductors  and  microcircuits. 

d.  yield  rates  (percent  of  items  which  pass  ATP  on 
first  try)  at  SRU  and  LRU  levels. 
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e.  Rework  hours  as  a  percent  of  total  assembly 
hours. 

4.  Delivered  product  quality  as  reflected  by" 

a.  Completed  item  inspection  and  system  test 
results . 

b.  Waivers,  deviations,  shortages,  unaccomplished 
tasks,  open/unr esolved  failures  as  documented 
in  acceptance  data  packages  and  DD  Form  250. 

c.  "Cannot  duplicate"  or  "retest  OK"  rate  on 
returned  units. 

5.  Innovations  to  enhanced  productivity  as  evidenced 
by: 

a.  Development/implementation  of  employee 
motivational  programs. 

b.  Effective  application  of  non-destructive 
evaluation  techniques. 

c.  Application  of  current  technologies  in 

inspection  and  test. 

d.  Application  of  current  technologies  in 

manufacturing  assembly  and  process  control. 

Product  Assurance 

1.  Management  commitment  as  evidenced  by  written 
policy,  organizational  recognition,  investment  of 
resources,  and  supportive  management  decisions. 
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3. 


4. 


Quality 

1. 

2. 

3. 

Quality 

1. 

2. 

3. 


Acceptance  testing  is  done  under  conditions  wmch 
stress  the  hardware  sufficiently  to  weed  out 
marginal  performers.  Such  stress  screening  is  done 
at  the  lowest  practical  levels. 

Failures  are  investigated  as  to  cause,  including 
physics  of  failure  analysis  when  appropriate,  and 
results  are  promptly  fed  back  into  design, 
manufacturing,  process  control,  inspection,  or 
test . 

Field  performance  exceeds  predicted  levels  for 
reliability,  growth,  availability,  and 

suppor  tabil ity . 

Evaluation  Areas  Suggested  by  Contractors 

of  Design 

Participation  in  Design  Review. 

Continued  Configuration  Management. 

Goodness  of  Design  (performance  oriented), 
of  Conformance 
Producibility/Reliabil ity. 

First  Article. 

Physical  Configuration  Audit  and  Functional 
Configuration  Audit. 
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Quality  of  Management 

1.  Management  of  Vendors  and  Suppliers. 

2.  Involvement  in  Increased  Productivity. 

3.  Independent  Quality  Assurance  reporting  to 

Executive  Division. 

4.  Properly  Manned  Organizational  Structure. 

5.  Quality  of  People  (knowledge  and  experience). 

6.  Trend  Analysis  and  Corrective  Action. 

Evaluation  Areas  Suggested  by  Government  Managers 

Quality  of  Design 

1.  Reliability  and  Maintainabililty  Criteria. 

a.  Mean  Time  Between  Failure. 

b.  Mean  Time  To  Repair. 

2.  Functional,  Performance,  and  Design  Specification. 

3.  Manufacturing  and  Test  Yields. 

4.  Engineering  Changes  Rate  (number  and  type  after  a 
specified  period). 

Quality  of  Conformance 

1.  Periodic  Teardown  Inspections. 

a.  Physical  Configuration  Audit. 

b.  Functional  Configuration  Audit. 

2.  Drawings  and  Specifications  requirements. 
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3.  Material  Review  Board  Activity. 


4.  Waivers,  Deviations,  and  Service  Reports. 

5.  Tests  and  Test  Procedures. 

Quality  of  Management 

1.  Status  of  the  Chief  of  Quality  and  his  ability  to 
affect  change. 

2.  Responsiveness  to  Field  Problems  and  tneir 
resolution . 

3.  Management  of  Vendors  and  Suppliers. 

4.  Approval  for  Changes  and  Substitutes. 

5.  Internal  Quality  Discipline. 

6.  Cost  of  Quality  (scrap,  rework,  repair). 

7.  Management  Involvement. 

8.  Employee  Motivation. 

9.  Corrective  Action. 
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